System and method for selectively obscuring a video signal

ABSTRACT

An object may be identified with a camera that is sensitive to both visible and invisible light. The camera receives a target of invisible light, either from an invisible light illuminator attached to the object, or via radiation from the object itself. The system processes the invisible light to position and size a halo with respect to the location of the object. The type of halo used may be determined by user selection. The system then modifies the visible component of the video signal from the camera to obscure the portion that lies outside the halo, thereby concealing the surroundings of the object.

BACKGROUND

1. Field of the Invention

The present invention relates generally to the field of videocommunication. More specifically, the present invention relates to asystem and method for automatically obscuring a portion of a videosignal during videoconferencing.

2. Description of Related Background Art

Videoconferencing is rapidly becoming the communication method-of-choicefor remote parties who wish to approximate face-to-face contact withoutthe time and expense of travel. As bandwidth limitations cease to becomea concern, a greater number of traditionally face-to-face events, suchas business meetings, family discussions, and shopping, may be expectedto take place through videoconferencing.

Unfortunately, despite the convenience, many people are still hesitantto participate in videoconferencing because they have little or nocontrol over what the other party sees. A user may wish to hide his orher environment, other people, or even a portion of his or her own bodyfrom the other person.

In response to such a need, schemes for background removal andreplacement have been developed. Known techniques typically usechromakeying (e.g., blue-screening or green-screening), backgroundsubtraction, motion analysis, or other similar computational methods todisplay a person while hiding the remainder of the scene. Unfortunately,such methods are problematic in a number of ways.

Often, known background alteration schemes are computationallyintensive. Hence, exceptionally powerful computing hardware is oftenrequired to avoid a noticeable delay between signal capture and signaltransmission. Furthermore, such schemes often require the use ofadditional equipment, such as colored screens, additional cameras,specialized lighting, and other devices that are cumbersome, expensive,and generally ill-suited to a home environment.

Even with the necessary equipment, such systems are often subject tounpredictable results. For example, in a typical chromakeying system, ifthe user is wearing an item of clothing that is the same color as thematte background, the clothing will be altered together with thebackground. Background subtraction systems may provide similar resultsif the person is wearing a similar color and/or pattern to that of thebackground image. Motion sensitive systems may improperly key onextraneous motion within the frame, and may even alter the image of auser that moves too little during videoconferencing.

Additionally, known systems often provide too little user control. Forexample, a user may wish to include a portion of the background withinthe image to show other people or objects. Conversely, the user may wishto exclude a portion of themselves from the image, particularly if theuser is not fully dressed or is engaged in activities, such as wrappinggifts, that they wish to hide from the other party. Known systemstypically lack such control; the user is unable to show more or lessthan the portion of the image selected by the applied computationalmethod.

Accordingly, what is needed is a system and method for reliably andconsistently obscuring a portion of a video signal. Such a system andmethod should be usable in videoconferencing applications, and shouldnot require the use of expensive, cumbersome, or inconvenient equipment.Furthermore, such a system and method should provide the user with atleast a general level of control over what is obscured.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-exhaustive embodiments of the invention are described with referenceto the figures, in which:

FIG. 1 is an illustration of a tracking system according to anembodiment of the invention;

FIG. 2 is an illustration of a pre-tracking frame from the camera ofFIG. 1;

FIG. 3 is an illustration of a centered frame from the camera of FIG. 1;

FIG. 4A is an illustration of a centered and zoomed frame from thecamera of FIG. 1;

FIG. 4B is an illustration of another centered and zoomed frame of thecamera of FIG. 1;

FIG. 5 is a schematic block diagram of one embodiment of avideoconferencing system in which the tracking system of FIG. 1 may beemployed;

FIG. 6 is a schematic block diagram of the camera of FIG. 1;

FIG. 7 is a schematic block diagram of another embodiment of a camerasuitable for tracking;

FIG. 8 is a schematic block diagram of one embodiment of a set top boxusable in connection with the videoconferencing system of FIG. 5;

FIG. 9 is a logical block diagram depicting the operation of thetracking system of FIG. 1;

FIG. 10 is a flowchart of a tracking method according to an embodimentof the invention;

FIG. 11 is a flowchart depicting one embodiment of a centering methodsuitable for the tracking method of FIG. 10;

FIG. 12 is a flowchart depicting another embodiment of a centeringmethod suitable for the tracking method of FIG. 10;

FIG. 13 is a flowchart depicting one embodiment of a zooming methodsuitable for the tracking method of FIG. 10;

FIG. 14 is a flowchart depicting another embodiment of a zooming methodsuitable for the tracking method of FIG. 10;

FIG. 15 is a plan view of an exemplary video communication system thatincorporates a plurality of cameras;

FIG. 16 is a block diagram of a database containing identitycharacteristics, identities, and settings;

FIG. 17 is a logical block diagram depicting one possible mode ofoperation of the video communication system of FIG. 15;

FIG. 18 is a flowchart depicting one embodiment of an identificationmethod usable in conjunction with the video communication system of FIG.15;

FIG. 19 is an illustration of a centered and zoomed frame from thecamera of FIG. 1, with an elliptical halo positioned around the head ofthe person;

FIG. 20 is an illustration of a centered frame from the camera of FIG.1, with a body-shaped halo positioned around the visible portion of theperson's body;

FIG. 21 is a logical block diagram depicting one possible mode ofoperation of the video communication system of FIG. 15, with theaddition of an obscuring subsystem that obscures at least a portion ofthe visible component of the video signal prior to transmission; and

FIG. 22 is a flowchart depicting one possible embodiment of a haloapplication method suitable for the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention solves the foregoing problems and disadvantages byproviding a system and method for selectively obscuring a portion of avideo signal during video communication. Of course, the described systemand method are usable in a wide variety of other contexts, includingsecurity, manufacturing, law enforcement, and the like. Hence, thepresent invention should not be construed as being limited tovideoconferencing.

In one implementation, an illuminator that projects a form of invisiblelight, such as infrared light, is attached to an object to be tracked.Where the object is a person, such an illuminator may be attached (by anadhesive or the like) to an article worn by the person, such a pair ofglasses, a shirt collar, a tie clip, etc. The illuminator may also beapplied directly to the skin of the person.

The illuminator may be a reflector that simply reflects invisible light.A stationary invisible light emitter emits invisible light (such asinfrared light) in the direction of the reflector. The invisible lightis then reflected back to a camera that detects both visible andinvisible light. In the alternative, the illuminator may be a portableinvisible light emitter that generates its own invisible light.

The camera provides a video signal with visible and invisiblecomponents. The invisible component is utilized by a tracking subsystemto center the field-of-view of the camera on the illuminator. Centeringmay be accomplished with a mechanical camera by physically panning andtilting the camera until the illuminator is in the center of thefield-of-view. The camera may alternatively be a software steerabletype, in which case centering is accomplished by cropping the cameraimage such that the illuminator is in the center of the remainingportion.

A zoom subsystem may utilize the invisible and/or the visible componentto “zoom,” or magnify, the field-of-view to reach a desiredmagnification level. As with tracking, such zooming may be accomplishedmechanically or through software, using mathematical calculation andalignment or stepwise adjustment.

A storage subsystem may obtain information from a user, or from anothersource. The storage subsystem may obtain identities of people or objectsthat may be present in the vicinity of the system. The storage subsystemmay also obtain associated identity characteristics for each identity,as well as settings according to which the system is to act in thepresence of each identity. The identities, identity characteristics, andsettings may be stored in a database for subsequent use.

An identification subsystem may utilize the invisible component. Theidentification subsystem may first extract all present identitycharacteristics from the invisible component. The identitycharacteristics are characteristics of the targets projected by theilluminators, such as shapes, intensities, frequencies, wavelengths, andthe like. The identification subsystem may then compare the identitycharacteristics with the database provided by the storage subsystem todetermine which identities correspond with the identity characteristics,i.e., which identities are present within the field-of-view of thecamera.

The identities may be conveyed to a setting control subsystem, whichagain references the database to determine which settings apply for eachof the identities present. The setting control subsystem may transmitcontrol signals to the tracking subsystem, the zoom subsystem, a videopreparation subsystem, and/or a plurality of external devices such asthermostats and alarm systems.

An obscuring subsystem may receive the visible component of the videosignal, and may apply a halo to the video signal. As described ingreater detail below, a “halo” is an invisible outline or stencil thatis superimposed upon the visible component of the video signal. In oneembodiment, the portion of the visible component outside the halo isobscured (e.g., blurred) or replaced, while the portion of the visiblecomponent inside the halo is not changed. The obscuring subsystem mayhave a halo selection module, a halo sizing module, a halo positioningmodule, and a signal modification module.

If desired, the halo selection module may receive the type of halo toapply as one of the settings from the database. Each user of the videocommunication system may then select his or own preferred halo type,which may include parameters such as shape, relative size, and relativeposition.

The halo sizing module may receive a distance between the camera and theilluminator from the zoom subsystem. The halo sizing module may thenestablish the size of the halo based on the distance, so that the halomaintains the same size in relation to the person when the person movestoward or away from the camera.

The halo positioning module may receive an object vector from thetracking subsystem. The halo positioning module may establish theposition of the halo based upon the object vector, so that the haloremains properly positioned with respect to the person as the personmoves within the field-of-view of the camera.

Once the halo type, size, and position have been determined, the signalmodification module may be used to apply the halo to the visiblecomponent of the video signal. As noted, the portion of the visiblecomponent that lies outside the halo may then be obscured. “Obscuring”may entail blurring or otherwise modifying the portion of the visiblecomponent, or replacing the portion of the visible component with asingle color or an unrelated image. The partially-obscured visiblecomponent may then be conveyed to a video presentation subsystem forformatting in preparation for transmission.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, appearancesof the phrases “in one embodiment” or “in an embodiment” in variousplaces throughout this specification are not necessarily all referringto the same embodiment.

Furthermore, the described features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments. In thefollowing description, numerous specific details are provided, such asexamples of programming, user selections, network transactions, databasequeries, database structures, etc., to provide a thorough understandingof embodiments of the invention. One skilled in the relevant art willrecognize, however, that the invention can be practiced without one ormore of the specific details, or with other methods, components,materials, etc. In other instances, well-known structures, materials, oroperations are not shown or described in detail to avoid obscuringaspects of the invention.

The following discussion makes particular reference to two-way videocommunication. However, those skilled in the art recognize that videocommunication typically includes two-way audio communication. Thus,where video communication and corresponding components are specificallyillustrated, audio communication and corresponding components may beimplied.

Referring to FIG. 1, one embodiment of a tracking system 100 accordingto the invention is shown. The object 110 may be inanimate, or may be aperson, animal, or the like. The object 110 may have an invisible lightilluminator 120, or illuminator 120, disposed on the object 110. As usedherein, “invisible light” refers to electromagnetic energy with anyfrequency imperceptible to the human eye. Infrared light mayadvantageously be used due to the ease with which it can be generatedand reflected; however, a wide variety of other electromagnetic spectramay also be utilized according to the invention, such as ultraviolet.

The illuminator 120 may take the form of a reflector that reflectsinvisible light generated by some other body. The illuminator 120 mayconsist, for example, of a solid body with a reflective side coated withor formed of a substance that reflects invisible light. Such a surfacemay be covered by glass or plastic that protects the surface and/orserves as a barrier to the transmission of electromagnetic energy ofundesired frequencies, such as those of the visible spectrum. An exampleof an illuminator may be a retro reflector, available from 3M and othermanufacturers.

The illuminator 120 may have an adhesive surface facing opposite thereflective surface; the adhesive surface may be used to attach theilluminator 120 to the object 110. Of course, the illuminator 120 couldalso be attached to the object 110 using any other attachment method,including a clip, clamp, adhesive, magnet, pin, or the like.

In the alternative, the illuminator 120 may take the form of a portableemitter that actively generates invisible light. Such an emitter may,for example, take the form of a specialized bulb, lens, or bulb/lenscombination connected to a portable power source such as a battery. Sucha portable emitter may then be used in much the same manner as areflector, i.e., disposed on an object to permit tracking. The portableemitter may therefore have an adhesive or any other attachmentmechanism.

In the event that the illuminator 120 is a reflector, a stationaryinvisible light emitter 130, or emitter 130, may be used to emitinvisible light toward the object 110. The emitter 130 may be embodied,for example, as an infrared emitter, such as an infrared LED, well knownto those skilled in the art. As another example, the emitter 130 maytake the form of an ultraviolet (UV) emitter.

The emitter 130 may be disposed upon the camera 140, on a front face ofset top box (STB) for an interactive television system, or in othersuitable locations. Furthermore, the invisible light emitter 130 mayreceive electrical power through a power cord 132 or battery (notshown), and may project invisible light 134 over a broad angle so thatthe object 110 can move through a comparatively large space without theilluminator 120 passing beyond the illuminated space.

Conventional light sources, including natural and artificial lighting,are also present and project visible light that is reflected by theobject 110. Such light sources are not illustrated in FIG. 1 to avoidobscuring aspects of the invention.

A portion 136 of the invisible light 134 may be reflected by theilluminator 120 to reach a camera 140. In one embodiment, the camera 140is sensitive to both visible light and invisible light of the frequencyreflected by the illuminator 120. The camera 140 may have a housing 142that contains and protects the internal components of the camera 140, alens 144 through which the portion 136 of the invisible light 134 isable to enter the housing 142, a base 146 that supports the housing 142,and an output cord 148 through which a video signal is provided by thecamera 140. Of course, the camera 140 may be configured in other wayswithout departing from the spirit of the invention. For instance, thecamera 140 may lack a separate housing and may be integrated withanother device, such as a set top box (STB) for an interactivetelevision system.

The video signal produced by the camera 140 may simply include a staticimage, or may include real-time video motion suitable forvideoconferencing. The video signal may also include audio information,and may have a visible component derived from visible light received bythe camera 140 as well as an invisible component derived from theportion 136 of the invisible light 134.

The object 110 may have a vector 150 with respect to the camera 140. Thevector 150 is depicted as an arrow pointing from the camera 140 to theobject 110, with a length equal to the distance between the object 110and the camera 140. A center vector 152 points directly outward from thecamera 140, into the center of a field-of-view 160 of the camera 140.

The field-of-view 160 of the camera 140 is simply the volume of spacethat is “visible” to the camera 140, or the volume that will be visiblein an output image from the camera 140. The field-of-view 160 may begenerally conical or pyramidal in shape. Thus, boundaries of thefield-of-view 160 are indicated by dashed lines 162 that form agenerally triangular cross section. The field-of-view 160 may bevariable in size if the camera 140 has a “zoom,” or magnificationfeature.

As described in greater detail below, the present invention provides asystem and method by which the center vector 152 can be automaticallyaligned with the object vector 150. Such alignment may take place inreal time, such that the field-of-view 160 of the camera 140 follows theobject 110 as the object 110 moves. Optionally, the camera 140 mayautomatically zoom, or magnify, the object 110 within the field-of-view160. The operation of these processes, and their effect on the visibleoutput of the camera 140, will be shown and described in greater detailin connection with FIGS. 2 through 4.

Referring to FIG. 2, an exemplary pre-tracking view 200 of visibleoutput, i.e., a display of the visible component of the video signal, isshown. Since the pretracking view 200 is taken from the point of view ofthe camera 140, a rectangular cross-sectional view of the field-of-view160 is shown. The field-of-view 160 is thus assumed to berectangular-pyramidal in shape; if the field-of-view 160 were conical,the view depicted in FIG. 2 would be circular.

In FIG. 2, a person 210 takes the place of the generalized object 110 ofFIG. 1. The camera 140 may be configured to track the person 210, or ifdesired, a head 212 of the person, while the person 210 moves. A secondperson 213 is also present. The camera 140 may also be used to track aninanimate object such as a folder 214 held by the person 213.Illuminators 220, 221 may be attached to the person 210 and the folder214, respectively, in order to facilitate tracking.

In the case of the person 210, the illuminator 220 may be affixed to anarticle worn by the person 210, such as a pair of glasses, a piece ofjewelry, a tie clip, or the like. Like the illuminator 120 of FIG. 1,the illuminator 220 may have a reflective side and a non-reflective sidethat can be attached through the use of a clip, clamp, adhesive, magnet,pin, or the like. The illuminator 220 may then be affixed to an objectsuch as a pair of glasses 222 or, in the alternative, directly to theperson 210. The illuminator 221 may be easily affixed to the folder 214in much the same fashion.

Indeed, if desired, an invisible light illuminator need not be a solidobject, but may be a paint, makeup, or other coating applicable directlyto an object or to the skin of the person 210. Such a coating needsimply be formulated to reflect the proper frequency of invisible light.The coating may even be substantially transparent to visible light.

The person 210, or the head 212 of the person 210, may have a desiredview 232, or an optimal alignment and magnification level for videocommunications. Similarly, the folder 214 may have a desired view 234.The illuminators 220, 221 may be positioned at the respective centers ofthe desired views 232, 234, so that the field-of-view 160 may be alignedwith such a desired view.

Each of the illuminators 220, 221 provides a “target,” or a bright spotwithin the invisible component of the video signal from the camera 140.Thus, each illuminator 220, 221 enables the camera 140 to determine thedirection in which the associated object vector 150 points. Once theobject vector 150 is determined, the tracking system 100 may proceed toalign the object vector 150 with the center vector 152.

More specifically, a center 240 of the field-of-view 160 is an end viewof the center vector 152 depicted in FIG. 1. In the view of FIG. 2, theilluminator 220 disposed on the person 210 is an end view of the objectvector 150. Thus, “tracking,” refers to motion of the field-of-view 160until the center 240 is superimposed on the illuminator 220.Consequently, the center 240 is to be moved along a displacement 242between the center 240 and the illuminator 220.

Such movement may be broken down into two separate dimensions: a pandisplacement 244 and a tilt displacement 246. The pan displacement 244represents the amount “panning,” or horizontal camera rotation, thatwould be required to align the center 240 with the illuminator 220. Thetilt displacement 246 represents the amount of “tilting,” or verticalcamera rotation, that would be required to align the center 240 with theilluminator 220.

Panning and tilting may be carried out by physically moving the camera140. More specifically, physical motion of the camera 140 may be carriedout through the use of a camera alignment subsystem (not shown) thatemploys mechanical devices, such as rotary stepper motors. Two suchmotors may be used: one that pans the camera 140, and one that tilts thecamera 140.

In the alternative, panning and tilting may be carried out by leavingthe camera 140 stationary and modifying the video signal. For example,panning and tilting may be performed in conjunction with zooming bycropping the video signal. The video signal is obtained by capturing asecond field-of-view (not shown) that covers a comparatively broad area.For example, a wide-angle, or “fish-eye” lens could be used for the lens144 of the camera 140 to provide a wide second field-of-view. The firstfield-of-view 160 is then obtained by cropping the second field-of-viewand correcting any distortion caused by the wide angle of the lens 144.

Panning and tilting without moving the camera 140 may be referred to as“software steerable” panning and tilting, although the subsystems thatcarry out the tracking may exist in software, hardware, firmware, or anycombination thereof. Software steerable panning and tilting will bedescribed in greater detail subsequently.

Referring to FIG. 3, a centered view 300 of visible output from thecamera 140 is shown. The field-of-view 160 has been panned and tiltedthrough mechanical or software steerable processing such that the center240 is aligned with the illuminator 220 on the person 210; consequently,tracking has been performed. The center 240 is not shown in FIG. 3 forclarity. The desired view 232 of the head 212 of the person 210 is nowcentered within the field-of-view 160. However, the field-of-view 160has not been resized to match the desired view 232; hence, no zoominghas occurred. “Centering,” as used herein, may not require precisepositioning of the head within the center 240 of the field-of-view 160.In the view of FIG. 3, the head 212 is positioned slightly leftward ofthe center 240 of the field-of-view 160. This is due to the fact thatthe person 210 is not looking directly at the camera 140; hence, theilluminator 220 is disposed toward the right side of the head 212, fromthe perspective of the camera 140. Consequently, the illuminator 220 isdisposed at the center 240 of the field-of-view 160, but the head 212 isslightly offset. Such offsetting is unlikely to seriously impedevideoconferencing unless the field-of-view 160 is excessively narrow.

Referring to FIG. 4A, a zoomed and centered view 400 of visible outputfrom the camera 140 is shown. The illuminator 220 is still centeredwithin the field-of-view 160, and the field-of-view 160 has beencollapsed to match the desired view 232, in which the head 212 appearslarge enough to read facial expressions during verbal communication withthe person 210. Consequently, both tracking (centering) and zooming havebeen performed.

As with tracking, zooming may be performed mechanically, or “optically.”Optical zooming typically entails moving the lens or lenses of thecamera to change the size of the field-of-view 160. Additionally, lensesmay be mechanically added, removed, or replaced to provide additionalzooming capability.

In the alternative, zooming may also be performed through software. Forexample, an image may be cropped and scaled to effectively zoom in onthe remaining portion. Such zooming may be referred to as software, or“digital” zooming.

The tracking and zooming functions have been illustrated as separatesteps for clarity; however, tracking need not be carried out prior tozooming. Indeed, tracking and zooming may occur simultaneously inreal-time as the person 210 moves within the field-of-view 160. The head212 of the person 210 may thus be maintained continuously centered atthe proper magnification level during video communication. The trackingsystem 100, or multiple such tracking systems, may be used in a widevariety of applications. As mentioned previously, videoconferencing issimply one application in which such tracking systems may findparticular application.

A similar process may be carried out with the folder 214, or with anyother object with an illuminator 220 or 221 attached. FIG. 4B depictsthe results of a similar process carried out with respect to theilluminator 221 attached to the folder 214.

Referring to FIG. 4B, a zoomed and centered view 450 of a differentsegment of visible output from the camera 140 is shown. The illuminator221 is now centered within the field-of-view 160, and the field-of-view160 has been collapsed to match the desired view 234 that contains thefolder 214 with the illuminator 221. Once again, both tracking(centering) and zooming have been performed. Through the use of softwaresteerable methods, the views of FIG. 4A and FIG. 4B may besimultaneously obtained from a single camera 140 by simply croppingdifferent portions of the visible component.

As shown, the illuminator 221 has a diamond shape, which is distinctfrom the elliptical shape of the illuminator 220 depicted in FIG. 4A.Thus, the tracking system 100 may distinguish between the illuminators220 and 221. Such identification is useful for a variety of reasons,which will be set forth in greater detail subsequently. The followingdiscussion, with reference to FIG. 5 through FIG. 14, assumes that thehead 212 of the person 210 is the object to be tracked.

Referring to FIG. 5, one embodiment of a videoconferencing system 500that may incorporate one or more tracking systems 100 is shown. In oneimplementation, the videoconferencing system 500 relies on acommunication subsystem 501, or network 501, for communication. Thenetwork 501 may take the form of a cable network, direct satellitebroadcast (DBS) network, or other communications network.

The videoconferencing system 500 may include a plurality of set topboxes (STBs) 502 located, for instance, at customer homes or offices.Generally, an STB 502 is a consumer electronics device that serves as agateway between a customer's television 504 and the network 501. Inalternative embodiments, an STB 502 may be embodied more generally as apersonal computer (PC), an advanced television 504 with STBfunctionality, or other customer premises equipment (CPE).

An STB 502 receives encoded television signals and other informationfrom the network 501 and decodes the same for display on the television504 or other display device, such as a computer monitor, flat paneldisplay, or the like. As its name implies, an STB 502 is typicallylocated on top of, or in close proximity to, the television 504.

Each STB 502 may be distinguished from other network components by aunique identifier, number, code, or address, examples of which includean Internet Protocol (IP) address (e.g., an IPv6 address), a MediaAccess Control (MAC) address, or the like. Thus, video streams and otherinformation may be transmitted from the network 501 to a specific STB502 by specifying the corresponding address, after which the network 501routes the transmission to its destination using conventionaltechniques.

A remote control 506 is provided, in one configuration, for convenientremote operation of the STB 502 and the television 504. The remotecontrol 506 may use infrared (IR), radio frequency (RF), or otherwireless technologies to transmit control signals to the STB 502 and thetelevision 504. Other remote control devices are also contemplated, suchas a wired or wireless mouse or keyboard (not shown).

For purposes of the following description, one STB 502, TV 504, remotecontrol 506, camera 140, and emitter 130 combination is designated alocal terminal 508, and another such combination is designated a remoteterminal 509. Each of the terminals 508, 509 is designed to providevideoconferencing capability, i.e., video signal capture, transmission,reception, and display.

The components of the terminals 508, 509 may be as shown, or may bedifferent, as will be appreciated by those of skill in the art. Forexample, the TVs 504 may be replaced by computer monitors, webpads,PDA's, computer screens, or the like. The remote controls 506 mayenhance the convenience of the terminals 508, 509, but are not necessaryfor their operation. As mentioned previously, the STB 502 may beconfigured in a variety of different ways. The camera 140 and theemitter 130 may also be reconfigured or omitted, as will be describedsubsequently.

Each STB 502 may be coupled to the network 501 via a broadcast center510. In the context of a cable network, a broadcast center 510 may beembodied as a “head-end”, which is generally a centrally-locatedfacility within a community where television programming is receivedfrom a local cable TV satellite downlink or other source and packagedtogether for transmission to customer homes. In one configuration, ahead-end also functions as a Central Office (CO) in thetelecommunication industry, routing video streams and other data to andfrom the various STBs 502 serviced thereby.

A broadcast center 510 may also be embodied as a satellite broadcastcenter within a direct broadcast satellite (DBS) system. A DBS systemmay utilize a small 18-inch satellite dish, which is an antenna forreceiving a satellite broadcast signal. Each STB 502 may be integratedwith a digital integrated receiver/decoder (IRD), which separates eachchannel, and decompresses and translates the digital signal from thesatellite dish to be displayed by the television 504.

Programming for a DBS system may be distributed, for example, bymultiple high-power satellites in geosynchronous orbit, each withmultiple transponders. Compression (e.g., MPEG) may be used to increasethe amount of programming that can be transmitted in the availablebandwidth.

The broadcast centers 510 may be used to gather programming content,ensure its digital quality, and uplink the signal to the satellites.Programming may be received by the broadcast centers 510 from contentproviders (CNN®, ESPN®, HBO®, TBS®, etc.) via satellite, fiber opticcable and/or special digital tape. Satellite-delivered programming istypically immediately digitized, encrypted and uplinked to the orbitingsatellites. The satellites retransmit the signal back down to everyearth-station, e.g., every compatible DBS system receiver dish atcustomers' homes and businesses.

Some broadcast programs may be recorded on digital videotape in thebroadcast center 510 to be broadcast later. Before any recorded programsare viewed by customers, technicians may use post-production equipmentto view and analyze each tape to ensure audio and video quality. Tapesmay then be loaded into a robotic tape handling systems, and playbackmay be triggered by a computerized signal sent from a broadcastautomation system. Back-up videotape playback equipment may ensureuninterrupted transmission at all times.

Regardless of the nature of the network 501, the broadcast centers 510may be coupled directly to one another or through the network 501. Inalternative embodiments, broadcast centers 510 may be connected via aseparate network, one particular example of which is the Internet 512.The Internet 512 is a “network of networks” and is well known to thoseskilled in the art. Communication over the Internet 512 is accomplishedusing standard protocols, such as TCP/IP (Transmission ControlProtocol/Internet Protocol) and the like. If desired, each of the STBs502 may also be connected directly to the Internet 512 by a dial-upconnection, broadband connection, or the like.

A broadcast center 510 may receive television programming fordistribution to the STBs 502 from one or more television programmingsources 514 coupled to the network 501. Preferably, television programsare distributed in an encoded format, such as MPEG (Moving PictureExperts Group). Various MPEG standards are known, such as MPEG-2,MPEG-4, MPEG-7, and the like. Thus, the term “MPEG,” as used herein,contemplates all MPEG standards. Moreover, other videoencoding/compression standards exist other than MPEG, such as JPEG,JPEG-LS, H.261, and H.263. Accordingly, the invention should not beconstrued as being limited only to MPEG.

Broadcast centers 510 may be used to enable audio and videocommunications between STBs 502. Transmission between broadcast centers510 may occur (i) via a direct peer-to-peer connection between broadcastcenters 510, (ii) upstream from a first broadcast center 510 to thenetwork 501 and then downstream to a second broadcast center 510, or(iii) via the Internet 512. For instance, a first STB 502 may send avideo transmission upstream to a first broadcast center 510, then to asecond broadcast center 510, and finally downstream to a second STB 502.

Each of a number of the STBs 502 may have a camera 140 connected to theSTB 502 and an emitter 130 positioned in close proximity to the camera140 to permit videoconferencing between users of the network 501. Morespecifically, each camera 140 may be used to provide a video signal of auser. Each video signal may be transmitted over the network 501 anddisplayed on the TV 504 of a different user. Thus, one-way ormultiple-way communication may be carried out over the videoconferencingsystem 500, using the network 501.

If desired, each of the STBs 502 may form the hub of a multi-cameranetwork. For example, a home or place of business may have a pluralityof cameras positioned in different rooms, or with differentfields-of-view within the same room. Separate stationary emitters may beprovided if the illuminators used are reflectors. Hence, a secondemitter 530 may be positioned in the vicinity of a second camera 540connected to the STB 502. A third camera 542 may also be used. Indeed,it may be desirable to use more additional cameras, depending on theintended application and the size and layout of the place in which theywill be used.

Furthermore, each of the STBs 502 may form a control center for variousdevices. For example, the STBs 502 may each be connected to implementssuch as a thermostat 550 and a security control panel 560 for an alarmsystem. Other devices, such as stereos, fans, humidifiers, automaticdoors and windows, conventional telephones, computers, webpads, lights,space heaters, and appliances are not shown, but may also be connectedto an STB 502. The present invention may utilize the cameras 140, 540,542 to identify users and control such devices according to thewhereabouts of each user. The identification and control aspects of theinvention will be discussed in greater detail subsequently.

Of course, the videoconferencing system 500 illustrated in FIG. 5 ismerely exemplary. Other types of devices and networks may be used withinthe scope of the invention.

Referring to FIG. 6, a block diagram shows one embodiment of a camera140 according to the invention. The camera 140 may receive both visibleand invisible light through the lens 144, and may process both types oflight with a single set of hardware to provide the video signal. Inaddition to the lens 144, the camera 140 may include a shutter 646, afilter 648, an image collection array 650, a sample stage 652, and ananalog-to-digital converter (ADC) 654.

As mentioned previously, if software steerable panning and tilting areto be utilized, the lens 144 may be a wide angle lens that has anangular field of, for example, 140 degrees. Using a wide angle lensallows the camera 140 to capture a larger image area than a conventionalcamera. The shutter 646 may open and close at a predetermined rate toallow the visible and invisible light into the interior of the camera140 and onto the filter 648.

The filter 648 may allow the image collection array 650 to accuratelycapture different colors. The filter 648 may include a static filtersuch as a Bayer filter, or may utilize a dynamic filter such as aspinning disk filter. Alternatively, the filter 648 may be replaced witha beam splitter or other color differentiation device. As yet anotheralternative, the camera 140 may be made to operate without any filter orother color differentiation device.

The image collection array 650 may included charge coupled device (CCD)sensors, complementary metal oxide semiconductor (CMOS) sensors, orother sensors that convert electromagnetic energy into readable imagesignals. If software steerable panning and tilting are to be used, thesize of the image collection array 650 may be comparatively large suchas, for example, 1024×768, 1200×768, or 2000×1000. Such a large sizepermits the image collection array 650 to capture a large image to formthe video signal from the comparatively large second field-of-view. Thelarge image can then be cropped and/or distortion-corrected to providethe properly oriented first field-of-view 160 without producing anoverly grainy or diminutive image.

The sample stage 652 may read the image data from the image collectionarray 650 when the shutter 646 is closed. The ADC 654 may then convertthe image data from analog to digital form to provide the video signalultimately output by the camera 140. The video signal may then betransmitted to the STB 502, for example, via the output cord 148depicted in FIG. 1 for processing and/or transmission. In thealternative, the video signal may be processed entirely by components ofthe camera 140 and transmitted from the camera 140 directly to thenetwork 501, the Internet 512, or other digital communication devices.

Those of skill in the art will recognize that a number of knowncomponents may also be used in conjunction with the camera 140. Forpurposes of explaining the functionality of the invention, such knowncomponents that may be included in the camera 140 have been omitted fromthe description and drawings.

Referring to FIG. 7, another embodiment of a camera 740 according to theinvention is depicted. Rather than processing visible and invisiblelight simultaneously with a single set of hardware, the camera 740 mayhave a visible light assembly 741 that processes visible light and aninvisible light assembly 742 that processes invisible light. The camera740 may also have a range finding assembly 743 that determines thelength of the object vector 150, which is the distance between thecamera 140 and the person 210.

The visible light assembly 741 may have a lens 744, a shutter 746, afilter 748, an image collection array 750, a sample stage 752, and ananalog-to-digital converter (ADC) 754. The various components of thevisible light assembly 741 may be configured in a manner similar to thecamera 140 of FIG. 6, except that the visible light assembly 741 neednot process invisible light. If desired, the lens 744 may be made toblock out a comparatively wide range of invisible light. Similarly, theimage collection array 750 may record only Visible light.

By the same token, the invisible light assembly 742 may have a lens 764,a shutter 766, a filter 768, an image collection array 770, a samplestage 772, and an analog-to-digital converter (ADC) 774 similar to thoseof the visible light assembly 741, but configured to receive invisiblerather than visible light. Consequently, if desired, the lens 764 may betinted, coated, or otherwise configured to block out all but thefrequencies of light reflected by the illuminators 220, 221. Similarly,the image collection array 770 may record only the frequencies of lightprojected by the illuminators 220, 221.

Ultimately, the visible light assembly 741 may produce the visiblecomponent of the video signal, and the invisible light assembly 742 mayproduce the invisible component of the video signal. The visible andinvisible components may then be delivered separately to the STB 502, asshown in FIG. 7, or merged within the camera 140 prior to delivery tothe STB 502. The visible and invisible light assemblies 741, 742 neednot be entirely separate as shown, but may utilize some common elements.For example, a single lens may be used to receive both visible andinvisible light, while separate image collection arrays are used forvisible and invisible light. Alternatively, a single image collectionarray may be used, but may be coupled to separate sample stages. Manysimilar variations may be made. As used herein, the term “camera” mayrefer to either the camera 140, the camera 740, or different variationsthereof.

The range finding assembly 743 may have a trigger/timer 780 designed toinitiate range finding and relay the results of range finding to the STB502. The trigger/timer 780 may be coupled to a transmitter 782 and areceiver 784. When triggered by the trigger/timer 780, the transmitter782 sends an outgoing pulse 792, such as an infrared or sonic pulse,toward the head 212 of the person 210. The outgoing pulse 792 bouncesoff the head 212 and returns in the form of an incoming pulse 794 thatcan be received by the receiver 784.

The trigger/timer 780 may measure the time differential betweentransmission of the outgoing pulse 792 and receipt of the incoming pulse794; the distance between the head 212 and the camera 740 isproportional to the time differential. The raw time differential or acalculated distance measurement may be transmitted by the trigger/timer780 to the STB 502. Determining the distance between the head 212 andthe camera 740 may be helpful in zooming the first field-of-view 160 tothe proper magnification level to obtain the desired view 232.

Numerous other camera embodiments may be used according to theinvention. Indeed, a more traditional analog camera may be used to readvisible and invisible light. Such an analog camera may provide an analogvideo signal that can be subsequently digitized, or may includeanalog-to-digital conversion circuitry like the ADC 754 and the ADC 774.For the sake of brevity, the following discussion assumes the use of thecamera 140.

If desired, the video signal may be processed outside the camera 140. Ifsoftware steerable panning and tilting is utilized, such processing mayinclude cropping and distortion correction of the video signal. If thecamera 140 is used as part of a videoconferencing system like thevideoconferencing system 500, the STB 502 may be a logical place inwhich to carry out such processing.

Referring to FIG. 8, there is shown a block diagram of physicalcomponents of an STB 502 according to an embodiment of the invention.The STB 502 may include a network interface 800 through which televisionsignals, video signals, and other data may be received from the network501 via one of the broadcast centers 510. The network interface 800 mayinclude conventional tuning circuitry for receiving, demodulating, anddemultiplexing MPEG-encoded television signals, e.g., digital cable orsatellite TV signals. In certain embodiments, the network interface 800may include analog tuning circuitry for tuning to analog televisionsignals, e.g., analog cable TV signals.

The network interface 800 may also include conventional modem circuitryfor sending or receiving data. For example, the network interface 800may conform to the DOCSIS (Data Over Cable Service InterfaceSpecification) or DAVIC (Digital Audio-Visual Council) cable modemstandards. Of course, the network interface and tuning functions couldbe performed by separate components within the scope of the invention.

In one configuration, one or more frequency bands (for example, from 5to 30 MHz) may be reserved for upstream transmission. Digital modulation(for example, quadrature amplitude modulation or vestigial sidebandmodulation) may be used to send digital signals in the upstreamtransmission. Of course, upstream transmission may be accomplisheddifferently for different networks 501. Alternative ways to accomplishupstream transmission include using a back channel transmission, whichis typically sent via an analog telephone line, ISDN, DSL, or othertechniques.

A bus 805 may couple the network interface 800 to a processor 810, orCPU 810, as well as other components of the STB 502. The CPU 810controls the operation of the STB 502, including the other componentsthereof. The CPU 810 may be embodied as a microprocessor, amicrocontroller, a digital signal processor (DSP) or other device knownin the art. For instance, the CPU 810 may be embodied as an Intel® x86processor. The CPU 810 may perform logical and arithmetic operationsbased on program code stored within a memory 820.

The memory 820 may take the form of random access memory (RAM), forstoring temporary data and/or read-only memory (ROM) for storing morepermanent data such as fixed code and configuration information. Thememory 820 may also include a mass storage device such as a hard diskdrive (HDD) designed for high volume, nonvolatile data storage.

Such a mass storage device may be configured to store encoded televisionbroadcasts and retrieve the same at a later time for display. In oneembodiment, such a mass storage device may be used as a personal videorecorder (PVR), enabling scheduled recording of television programs,pausing (buffering) live video, etc.

A mass storage device may also be used in various embodiments to storeviewer preferences, parental lock settings, electronic program guide(EPG) data, passwords, e-mail messages, and the like. In oneimplementation, the memory 820 stores an operating system (OS) for theSTB 502, such as Windows CE® or Linux®; such operating systems may bestored within ROM or a mass storage device.

The STB 502 also preferably includes a codec (encoder/decoder) 830,which serves to encode audio/video signals into a network-compatibledata stream for transmission over the network 501. The codec 830 alsoserves to decode a network-compatible data stream received from thenetwork 501. The codec 830 may be implemented in hardware, firmware,and/or software. Moreover, the codec 830 may use various algorithms,such as MPEG or Voice over IP (VoIP), for encoding and decoding.

In one embodiment, an audio/video (AN) controller 840 is provided forconverting digital audio/video signals into analog signals forplayback/display on the television 504. The AN controller 840 may beimplemented using one or more physical devices, such as separategraphics and sound controllers. The AN controller 840 may includegraphics hardware for performing bit-block transfers (bit-blits) andother graphical operations for displaying a graphical user interface(GUI) on the television 504.

The STB 502 may also include a modem 850 by which the STB 502 isconnected directly to the Internet 512. The modem 850 may be a dial-upmodem connected to a standard telephone line, or may be a broadbandconnection such as cable, DSL, ISDN, or a wireless Internet service. Themodem 850 may be used to send and receive various types of information,conduct videoconferencing without the network 501, or the like.

A camera interface 860 may coupled to receive the video signal from thecamera 140. The camera interface 860 may include, for example, auniversal serial bus (USB) port, a parallel port, an infrared (IR)receiver, an IEEE 1394 (“firewire”) port, or other suitable device forreceiving data from the camera 140. The camera interface 860 may alsoinclude decoding and/or decompression circuitry that modifies the formatof the video signal.

Additionally, the STB 502 may include a wireless receiver 870 forreceiving control signals sent by the remote control 506 and a wirelesstransmitter 880 for transmitting signals, such as responses to usercommands, to the remote control 506. The wireless receiver 870 and thewireless transmitter 880 may utilize infrared signals, radio signals, orany other electromagnetic emission.

A compression/correction engine 890 and a camera engine 892 may bestored in the memory 820. The compression/correction engine 890 mayperform compression and distortion compensation on the video signalreceived from the camera 140. Such compensation may permit a wide-angle,highly distorted “fish-eye” image to be shown in an undistorted form.The camera engine 892 may accept and process user commands relating tothe pan, tilt, and/or zoom functions of the camera 140. A user may, forexample, select the object to be tracked, select the zoom level, orother parameters related to the operation of the tracking system 100.

Of course, FIG. 8 illustrates only one possible configuration of an STB502. Those skilled in the art will recognize that various otherarchitectures and components may be provided within the scope of theinvention. In addition, various standard components are not illustratedin order to avoid obscuring aspects of the invention.

Referring to FIG. 9, a logical block diagram 900 shows one possiblemanner in which light and'signals may interact in the tracking system100 of FIG. 1. The illustrated steps/components may be implemented inhardware, software, or firmware, using any of the components of FIG. 8,alone or in combination. While various components are illustrated asbeing disposed within a STB 502, those skilled in the art will recognizethat similar components may be included within the camera, itself.

As described previously, if the illuminator 220 is a reflector, thestationary emitter 130 emits invisible light 134 that is reflected bythe illuminator 220. If the illuminator 220 is a portable emitter, theilluminator 220 may then generate invisible light independent of astationary emitter. Ambient light sources 930 have not been shown inFIG. 1 for clarity; the ambient light sources 930 may include the sun,incandescent lights, fluorescent lights, or any other source thatproduces visible light 934. The visible light 934 reflects off of theobject 212 (e.g., head), and possibly the illuminator 220.

Both visible and invisible light are reflected to the camera 140, whichproduces a video signal with a visible light component 940 and aninvisible light component 942. The visible light component 940 and theinvisible light component 942 are conveyed to the STB 502. If a camerasuch as the camera 740 is used, the camera 740 may also transmit thedistance between the camera 740 and the object 212, which is determinedby the range finding assembly 743, to the STB 502.

The invisible light component 942 may be processed by a trackingsubsystem 950 that utilizes the invisible light component 942 to orientthe field-of-view 160. For example, the tracking subsystem 950 may movethe field-of-view 160 from that shown in FIG. 2 to that shown in FIG. 3.

The tracking subsystem 950 may have a vector calculator 960 thatdetermines the direction in which the object vector 150 points. Such adetermination may be relatively easily made, for example, by determiningwhich pixels of the digitized invisible light component 942 contain thetarget reflected by the illuminator 220.

The vector calculator 960 may, for example, measure luminance values orthe like to determine which pixels correspond to the illuminator 220.The target reflected by the illuminator 220 can be expected to be thebrightest portion of the invisible component 942. The frequency andintensity of the invisible light emitted by the emitter 130 may beselected to ensure that the brightest invisible light received by thecamera 140 is that reflected by the illuminator 220.

Alternatively, the tracking subsystem 950 may determine the location ofthe illuminator 220 through software such as an objectivicationalgorithm that analyzes motion of the illuminator 220 with respect tosurrounding objects. Such an objectivication algorithm may separate thefield-of-view 160 into “objects,” or portions that appear to movetogether, and are therefore assumed to be part of a common solid body.Thus, the tracking subsystem 950 may resolve the illuminator 220 intosuch an object, and perform tracking based on that object. As oneexample, an algorithm such as MPEG-4 may be used.

Assuming the tracking subsystem 950 uses vector analysis, the vectorcalculator 960 may provide the object vector 150 to a field-of-vieworientation module 962. The field-of-view orientation module 962 maythen center the camera 140 on the object 212 (e.g., align the centervector 152 with the object vector 150).

Thus, the field-of-view orientation module 962 may perform the centeringoperation shown in FIG. 2 to align the center 240 of the field-of-view160 with the target reflected by the illuminator 220. The field-of-vieworientation module 962 may, for example, determine the magnitudes of thepan displacement 244 and the tilt displacement 246, and perform theoperations necessary to pan and tilt the field-of-view 160 by theappropriate distances. As mentioned previously, panning and tilting maybe performed mechanically, or through software.

The magnitudes of the pan and tilt displacements 244, 246 do not dependon the distance between the object 212 and the camera 140. Consequently,the tracking subsystem 950 need not determine how far the object 212 isfrom the camera 140 to carry out tracking. A two-dimensional objectvector 150, i.e., a vector with an unspecified length, is sufficient fortracking.

As an alternative to the analytical tracking method described above, thetracking subsystem 950 may perform tracking through trial and error. Forexample, the tracking subsystem 950 need not determine the object vector150, but may simply determine which direction the field-of-view 160 mustmove to bring the object 212 nearer the center 240. In other words, thetracking subsystem 950 need not determine the magnitudes of the pan andtilt displacements 244, 246, but may simply determine their directions,i.e., up or down and left or right. The field-of-view 160 may then berepeatedly panned and/or tilted by a preset or dynamically changingincremental displacement until the object 212 is centered within thefield-of-view 160.

The STB 502 may also have a zoom subsystem 952 that widens or narrowsthe field-of-view 160 to the appropriate degree. The zoom subsystem 952may, for example, modify the field-of-view 160 from that shown in FIG. 3to that shown in FIG. 4.

Since the camera 140 shown in FIG. 9 does not have range findinghardware, the zoom subsystem 952 may have a range finder 970 thatdetermines a distance 972 between the camera 140 (or the STB 502) andthe object 212. The range finder 970 may be configured in a mannersimilar to the range finding assembly 743 of the camera 740, with atrigger/timer, transmitter, and receiver (not shown) that cooperate tosend and receive an infrared or sonic pulse and determine the distancebased on the lag between outgoing and incoming pulses.

If a camera with a range finding assembly 743 or other range findinghardware, such as the camera 740, were to be used in place of the camera140, the STB 502 may not require a range finder 970. The tracking system100 may alternatively determine the distance between the camera 140 andthe object 212 through software such as an objectivication algorithm(e.g., as in MPEG4) that determines the size of the head 212 within thefield-of-view 160 based on analyzing motion of the head 212 with respectto surrounding objects.

The distance 972 obtained by the range finder 970 may be conveyed to amagnification level adjustment module 974, which may use the distance972 to zoom the field-of-view 160 to an appropriate magnification level.The magnification level may be fixed, intelligently determined by themagnification level subsystem 974, or selected by the user.

In any case, the magnification level may vary in real-time such that theobject 212 always appears to be the same size within the field-of-view160. Such zooming may be performed, for example, through the use of asimple linear mathematical relationship between the distance 972 and thesize of the field-of-view 160. More specifically, the ratio of objectsize to field-of-view size may be kept constant.

For example, when the head 212 of the person 210 moves away from thecamera 140, the magnification level adjustment module 974 may narrow thefield-of-view 160 (e.g., “zoom in”) so that the ratio of sizes betweenthe head 212 and the field-of-view 160 remains the same. Thefield-of-view size refers to the size of the rectangular area processedby the camera, such as the views of FIG. 2, FIG. 3, and FIG. 4. If thehead 212 moves toward the camera 140, the field-of-view 160 may bebroadened (e.g., “zoomed out”) to maintain the same ratio. Thus, thefacial features of the person 210 will still be easily visible when theperson 210 moves toward or away from the camera 140.

In the alternative to the analytical zooming method described above,zooming may also be performed through trial and error. For example, themagnification level adjustment module 974 may simply determine whetherthe field-of-view 160 is too large or too small. The field-of-view 160may then be repeatedly broadened or narrowed by a preset increment untilthe field-of-view 160 is zoomed to the proper magnification level, i.e.,until the ratio between the size of the object 212 and the size of thefield-of-view 160 is as desired.

The visible light component 940 of the video signal from the camera 140may be conveyed to a video preparation subsystem 954 of the STB 502. Thevideo preparation subsystem 954 may have a formatting module 980 thattransforms the visible light component 940 into a formatted visiblecomponent 982 suitable for transmission, for example, to the broadcastcenter 510 to which the STB 502 is connected. The formatted visiblecomponent 982 may also be displayed on the TV 504 connected to the STB502, for example, if the person 210 wishes to verify that the camera 140is tracking his or her head 212 properly.

The field-of-view orientation module 962 and the magnification leveladjustment module 974 determine the orientation and zoom level of theformatted visible light component 982. In the case of mechanicalpanning, tilting, and zooming, the camera 140 may be controlled by thefield-of-view orientation module 962 and the magnification leveladjustment module 974. Thus, the visible light component 940 wouldalready be properly oriented and zoomed.

However, the logical block diagram 900 of FIG. 9 assumes that panning,tilting, and zooming are managed through software. Thus, thefield-of-view orientation module 962 and the magnification leveladjustment module 974 may interact directly with the formatting module980 to modify the visible light component 940. More specifically, theformatting module 980 may receive instructions from the field-of-vieworientation module 962 and the magnification level adjustment module 974to determine how to crop the visible light component 940. Aftercropping, the formatted visible light component 982 provides a centeredand zoomed image.

The formatted visible component 982 may be conveyed over the network 501to the remote terminal 509, which may take the form of another STB 502,TV 504, and/or camera 140 combination, as shown in FIG. 5. A user at theremote terminal 509 may view the formatted visible component 982, andmay transmit a visible component of a second video signal captured bythe remote terminal 509 back to the local terminal 508 for viewing onthe TV 504 of the local terminal 508. Thus, the users of the local andremote terminals 508, 509 may carry out two-way videoconferencingthrough the use of the communication subsystem 501, or network 501.

If desired, software steerable technology may be used to provide asecond formatted visible light component (not shown) of a differentobject. For example, the visible light component 940 of the video signalfrom the camera 140 may be cropped a first time to provide the desiredview 232 of the head 212 of the person 210, as shown in FIG. 4. Thedesired view 232 may be formatted to form the formatted visiblecomponent 982. The visible light component 940 may be cropped a secondtime to provide the desired view 234 of the folder 214. The desired view234 of the folder 214 may be formatted to form the second formattedvisible light component 982.

In such a fashion, a plurality of additional cropped subsets of thevisible light component 940 may be provided. Each cropped subset may besent to a different remote terminal 509, for example, if multipleparties wished to see different parts of the view of FIG. 2. Thus,multiple objects can be tracked and conveyed over the network 501 with asingle camera 140. Of course, one cropped subset could be displayed onthe TV 504 of the local terminal 508 or recorded for future playback.

The tracking system 100 also may perform other functions aside fromvideoconferencing. For example, the tracking system 100 may be used tolocate articles for a user. An illuminator 220 or 221 may be attached toa set of car keys, the remote control 506, or the like, so that a usercan activate the tracking system 100 to track the car keys or the remotecontrol 506.

An object may, alternatively, be equipped with an active emitter thatgenerates invisible light that can be received by the camera 140. Theremote control 506 may, for example, emit invisible light, eitherautonomously or in response to a user command, to trigger tracking anddisplay of the current whereabouts of the remote control 506 on the TV504.

An illuminator 220, 221 may also be disposed on a child to be watched bya parent or other caregiver. A user may then use the tracking system 100to determine the current location of the child, and display the child'sactivities on the TV 504. Thus, the tracking system 100 can be used in awide variety of situations besides traditional videoconferencing.

Referring to FIG. 10, one possible embodiment of a tracking method 1000that may be carried out in conjunction with the tracking system 100 isdepicted. The method 1000 assumes that the object 212 is to be tracked.The illuminator 220 may first be attached 1010 to the object 212. Suchattachment may be accomplished through any known attachment mechanism,including clamps, clips, pins, adhesives, or the like.

Invisible light 134 may then be emitted 1020 such that the invisiblelight 134 enters the field-of-view 160 and impinges against theilluminator 220. The illuminator 220 projects 1030 the portion 136 ofthe invisible light 134 to the camera 140. The camera 140 captures 1040a first video signal that includes the visible component 940 derivedfrom visible light received by the camera 140 and the invisiblecomponent 942 derived from the portion 136 of invisible light receivedby the camera 140. A “video signal” includes multiple frames captured ata certain rate; however, the step of capturing 1040 the first videosignal may be interpreted as capturing a single frame. Hence the method1000 may be carried out on a frame-by-frame basis.

The field-of-view 160 is then moved 1050 or oriented, for example, bythe tracking subsystem 950 to center the object 212 within the invisiblecomponent 942. The size of the field-of-view 160 may be adjusted by thezoom subsystem 952 to obtain the desired zoom factor.

Since the head 212 of the person 210 can be expected to move aboutwithin the field-of-view 160, tracking and zooming may be carried outcontinuously until centering and zooming are no longer desired. Iftracking is to continue 1070, the steps from emitting 1020 invisiblelight through adjusting 1060 the magnification level may be repeatedcontinuously. If there is no further need for tracking and zooming,i.e., if videoconferencing has been terminated or the user has otherwiseselected to discontinue zooming and tracking, the tracking method 1000may terminate.

For each of the steps of moving 1050 the field-of-view 160 and adjusting1060 the magnification level of the field-of-view 160, the trackingsystem 100 may perform multiple tasks. Such tasks will be outlined ingreater detail in connection with FIGS. 11 and 12, which provide twoembodiments for moving 1050 the field-of-view 160, and FIGS. 13 and 14,which provide two embodiments for adjusting 1060 the magnification levelof the field-of-view 160.

Referring to FIG. 11, moving 1050 the field-of-view 160 may includedetermining 1110 the location of the target reflected by the illuminator220 within the field-of-view 160. The object vector 150 may then becalculated 1120, for example, by the vector calculator 960. Thefield-of-view 160 may then be panned and tilted 1130 to align the centervector 152 of the field-of-view 160 with the object vector 150.

Referring to FIG. 12, an alternative embodiment of a centering method1200 is depicted, which may operate in place of the method 1050described in FIG. 11. The method 1050 of FIG. 11 may be referred to asanalytical, while the method 1200 utilizes trial and error.

The centering method 1200 may commence with determining 1210 thedirection the target, or the object 212, is displaced from the center240 of the field-of-view 160. The field-of-view 160 may then be moved1220, or panned and tilted, so that the center 240 is brought closer tothe target provided by the illuminator 220, or the object 212. If thetarget is not yet centered, the steps of determining 1210 the directionto the target and moving 1220 the field-of-view 160 may be repeateduntil the target is centered, or within a threshold distance of thecenter 240 of the field-of-view 160.

Referring to FIG. 13, adjusting 1060 the magnification level of thefield-of-view 160 may commence with determining 1310 the distance 972between the object 212 and the camera 140. Determining 1310 the distancemay be carried out by the range finder 970, or by a range findingassembly 743 if a camera such as the camera 740 is used. The desiredmagnification level of the field-of-view 160 may then be calculated 1320using the distance 972, for example, by maintaining a constant ratio ofthe distance 972 to the size of the field-of-view 160. The camera maythen be zoomed 1330 until the desired magnification level has beenachieved.

Referring to FIG. 14, an alternative embodiment of a zooming method 1400is depicted, which may operate in place of the method 1060 described inFIG. 13. Like the method 1050 of FIG. 11, the method 1060 of FIG. 13 maybe referred to as analytical, while the method 1400 utilizes trial anderror, like the method 1200.

The method 1400 may first determine 1410 whether the magnification levelis too large or too small, i.e., whether the object 212 appears toolarge or too small in the field-of-view 160. The magnification level maythen be changed 1420 incrementally in the direction required to approachthe desired magnification level. If the best (i.e., desired)magnification level has not been obtained 1430, the method 1400 mayiteratively determine 1410 in which direction such a change is necessaryand change 1420 the magnification level in the necessary direction,until the desired magnification level is obtained.

The methods presented in FIGS. 10 through 14 may be utilized with anumber of different embodiments besides those explicitly described inthe foregoing examples. Furthermore, those of skill in the art willrecognize that other methods may be used to carry out tracking andzooming according to the invention.

For example, the invisible light produced by a normal human body may beused in place of the illuminator 220. The human body radiateselectromagnetic energy within the infrared spectrum; consequently, thecamera 140 or 540 may receive invisible light from the person 210without the aid of any emitter or reflector.

Tracking may be performed by determining the location of a “hot spot,”or area of comparatively intense infrared radiation, such as the head212. The forehead and eyes tend to form such a hot spot; hence, trackingbased on infrared intensity may provide easy centering on the eyes ofthe person. Other areas of relatively higher infrared intensity (e.g.,the chest) are typically covered by clothing. Hence, for applicationssuch as videoconferencing, tracking based on the intensity of infraredradiation from the human body provides a technique for centering thehead 212 within the field-of-view 160.

In certain configurations, a high-speed camera 140 may capture videoframes at greater than the normal rate, e.g., 60 frames per second (fps)instead of 30 fps. The illuminator 220 may generate or reflect invisiblelight 134 at periodic intervals such that the target does not appear inevery frame of the video signal. For example, in the above example, thetarget would only appear in alternate frames. As a result, the trackingsubsystem may distinguish a potential target from other sources ofinvisible light by determining that the potential target does not appearin every frame (e.g., does not appear in adjacent frames in the aboveexample). The frames without the target may be used for videocommunication, while the frames with the target may be used for trackingpurposes.

The advantage of this approach is that it allows the system todistinguish between hot spots generated by the illuminator 220 and hotspots generated by other sources, e.g., lamps, sunlight, etc. The systemneed only look for a target of invisible light 134, for example, in theeven frames that does not also exist in the odd frames.

In an alternative configuration, tracking may be performed by locatingan area that emits a comparatively specific infrared frequency. Ifdesired, the camera 140 or 540 and/or STB 502 may be calibrated to theindividuals with which they will be used. Thus, the camera 140 or 540will be able to perform tracking despite ordinary variations in bodytemperature from one person to the next.

An objectivication algorithm may also be used in conjunction withtracking based on the infrared radiation of the human body. Morespecifically, objectivication may be utilized to resolve the invisiblecomponent 942 into one or more people based on the shapes and/or motionof the infrared radiation received. Thus, the locations of people withinthe field-of-view 160 can be determined without the use of a reflectoror emitter.

Those of skill in the art will recognize that tracking may also beaccomplished in a number of ways within the scope of the invention. Forexample, low power microwave radiation may be emitted by an emittersimilar to the emitter 130 of FIG. 1. Invisible light within themicrowave frequency band may be somewhat more readily distinguished fromambient light, such as electromagnetic emissions from the sun,artificial lights, or other warm objects. The light produced by suchambient sources may be mostly infrared or visible. Hence, the use ofmicrowave radiation may enable more effective tracking by reducingambient interference. Microwave radiation may be read and processed insubstantially the same manner as described above.

Furthermore, regardless of the frequency of light detected, additionalprocessing may be carried out to distinguish between objects to betracked and surrounding objects. For example, through a method such asDoppler detection, differentials between emitted wavelengths andreceived wavelengths may be used to determine whether an object ismoving toward or away from the camera. Objects in motion, such aspeople, may therefore project light with a frequency shifted somewhatfrom the frequency of the emitted light. Conversely, stationary objectsmay be assumed to project a consistent frequency. Thus, a moving objectmay be distinguished from other changes in electromagnetic emission,such as changing sunlight patterns.

Referring to FIG. 15, a plan view is shown of a video communicationsystem 1500 according to an embodiment of the invention. A plurality ofwalls 1502 are shown to denote an interior space, such as rooms of ahome or place of business. The walls 1502 are shaped to form a firstalcove 1504 and a second alcove 1506 that do not have directline-of-sight to each other. Hence, the first camera 140, first emitter130, and STB 502 may be positioned within the first alcove 1504, whilethe second camera 540 and second emitter 530 are positioned in thesecond alcove 1506. An overlap region 1508 is visible from both of thealcoves 1504, 1506.

The cameras 140, 540 may be networked together, with the STB 502 as thehub of the network. More specifically, the output cord 148 of the camera140 may be connected directly to the STB 502, while the output cord 148of the camera 540 is connected to the STB 502 via data lines (not shown)positioned within the walls 1502. Such data lines may take the form ofRJ-45 (Category 5) cables, coaxial data cables, or other cables commonlypresent in homes and offices for computer networking purposes. Ofcourse, wireless networking may also be used within the scope of theinvention.

The output cord 148 of the second camera 540 may thus simply be pluggedinto a networking jack 1520. The STB 502 may have a data line 1522plugged into a similar networking jack 1520, through which the STB 502receives data from the second camera 540 and any other cameras connectedto the data lines within the walls 1502.

The field-of-view 160 of the first camera 140 extends along the firstalcove 1504; once again, boundaries of the field-of-view 160 aredepicted by the dashed lines 162. A field-of-view 1560 of the secondcamera 540 extends along the second alcove 1506, as depicted by thedashed lines 1562. As shown, the people 210, 213 (shown as circles) areinitially located within the field-of-view 160 of the first camera 140.As with previous figures, the illuminator 220 is disposed on the person210, and the illuminator 221 is disposed on the person 213. Again, theilluminator 220 is elliptical in shape, while the illuminator 221 isdiamond-shaped. As described in greater detail below, characteristicsother than shapes may be used to differentiate between the illuminators220, 221.

If the person 210 is engaged in videoconferencing, it may be desirableto have the camera 140 track and zoom in on the person 210. Thevideoconferencing system 1500 thus needs to be able to determine whichof the illuminators 220, 221 to follow, and which to ignore. As aresult, it is desirable to enable the system 1500 to distinguish betweenthe illuminators 220, 221 and keep track of which person 210, 213 is tobe followed.

Such an identification function may also be used to enable automaticcontrol of other implements unrelated to videoconferencing. For example,as mentioned previously, the STB 502 may be connected to other devicessuch as a thermostat 550 and a security control panel 560.

As shown, the thermostat 550 is located in the first alcove 1504, and isdirectly connected to the STB 502 by a data line 1522. The securitycontrol panel 560 is positioned in the overlap region 1508, and isconnected to a networking jack 1520 by a data line 1522. The securitycontrol panel 560 is thus connected to the STB 502 via the data lineswithin the walls 1502, in a manner similar to the second camera 540.

Such devices may be automatically controlled by the STB 502 based on thelocation of the people 210, 213. For example, when a person 210 or 213enters the first alcove 1504, the system 1500 may detect their presenceand adjust the temperature setting of the thermostat 550 according totheir preference. Similarly, when a person 210 or 213 enters the secondalcove 1506 or the overlap region 1508, the system 1500 may detect theirpresence and transmit signals to the security control panel 560 todeactivate alarm systems.

Furthermore, the system 1500 may beneficially utilize the illuminators220, 221 to automatically control the cameras 140, 540. For example, theperson 210 may move from the first alcove 1504 to the second alcove1506, in the direction shown by the arrow 1570 to reach the positionshown by the person 210 and illuminator 220 in phantom. In such a case,it may be desirable to switch video reception from the first camera 140to the second camera 540. Such switching may beneficially occur withoutthe explicit control of the person 210; hence, videoconferencing maycontinue naturally and without interruption while the person 210 moves.

Consequently, it would be desirable for the videoconferencing system1500 to automatically identify objects, including people, within thefields-of-view 160, 1560. Such identification may be used to provideautomatic tracking, zooming, and control of other functions based on thelocation of key objects. Furthermore, it would be desirable for thesystem 1500 to automatically “handoff” video transmission between thecameras 140, 540 based on the location of the object that is the subjectof videoconferencing, i.e., the person 210 in the example of FIG. 15.

Identification and handoff may be carried out independently, or inconjunction with the intelligent tracking and/or zooming methodsdescribed previously. Exemplary systems and methods for accomplishingidentification and handoff will be described in connection with FIGS.16, 17, and 18, as follows.

Referring to FIG. 16, one embodiment of a database 1600 is shown. Thedatabase 1600 may be stored within the STB 502, for example, within thememory 820. The database 1600 may have a plurality of identitycharacteristics 1602 that correspond to properties of the targetsprojected by the illuminators 220, 221. The identity characteristics aresimply properties that can be recognized through the use of knowncomputerized matching techniques such as pattern recognition and opticalcharacter recognition (OCR) algorithms.

Such algorithms typically segment an image, for example, into a grid,and analyze the patterns formed by adjacent segments. Some algorithmsutilize artificial neural network (ANN) technology. Some utilize“connected component analysis,” or a method of grouping portions of animage into components and then analyzing their relationship with eachother. In any case, most such algorithms maintain a stored list ofknown, or “learned”, templates, with which shapes of the image are to becompared. The identity characteristics 1602 may thus be known templatesfor pattern recognition.

As shown, the identity characteristics 1602 may be shapes of theilluminators 220, 221, such as the ellipse and diamond depicted in FIGS.4A, 4B, and 15. The shapes may be selected for easy differentiation bycomputer; thus, simple rounded or polygonal shapes may advantageously beused. The shapes may also be selected such that no two are alike whenrotated or angled. Consequently, it may be helpful to avoid attemptingto distinguish between a circle and an ellipse, or a square and adiamond. Polygonal shapes may be beneficial in that a computer mayrelatively easily count the points or flat segments of the shape; suchan analysis does not depend greatly on the angle or rotationalorientation of the shape.

Of course, the identity characteristics 1602 need not be shapes, but maybe based on a variety of properties of the targets projected by theilluminators 220, 221. For example, the identity characteristics 1602may be wavelengths of invisible light received from the illuminators220, 221. The illuminators 220, 221 may simply be tuned to projectdifferent frequencies.

Alternatively, the identity characteristics 1602 may be intensities ofthe invisible light received from the illuminators 220, 221. Theilluminators 220, 221 may thus be configured to project light withdifferent intensities. For example, if the illuminators 220, 221 arereflectors, they may have comparatively different reflectivities. If theilluminators 220, 221 are portable emitters, they may operate atdifferent power levels.

The identity characteristics 1602 may also be sizes of the targetsprojected by the illuminators 220, 221. Without knowing the range toeach illuminator 220, 221, it may be impossible to determine whether asize difference is the result of different illuminator sizes ordifferent ranges. Hence, size determination may only be usable insituations in which the illuminators 220, 221 are kept generallyequidistant from the cameras 140, 540, or in which the range to eachilluminator 220, 221 is known. For example, if a range finder 970 suchas the range finding assembly 743 of FIG. 7 is present within the zoomsubsystem 952, the ranges to the illuminators 220, 221 may be known, andtherefore accounted for to enable identification based on size.

Alternatively, the identity characteristics 1602 may be patterns orfrequencies of variation of the invisible light received from theilluminators 220, 221. For example, if the illuminators 220, 221 areportable emitters, they may each provide a time-varied, or “pulsing,”pattern of infrared light that can be received and interpreted by thecameras 140, 540. The variation in patterns may simply be a variation inpulsing frequency. Alternatively, other differences such as varyingpulse durations or amplitudes may be introduced.

The identity characteristics 1602 may also be derived from the presenceof multiple illuminators. For example, each identity characteristic 1602may be a pattern of targets received from a set of illuminators 220 or221. The pattern may include shapes positioned with a specifiedseparation and relative orientation. Alternatively, the pattern maysimply specify a composition of shapes, for example, 30% squares, 50%circles, and 20% triangles.

The use of multiple shapes may be advantageous in that the shapes can beminiaturized and embedded in a fluid or powder. Thus, for example,shapes of the specified composition may be included in an inconspicuousmakeup or lotion. A user may then simply apply the customized makeup orlotion to the part of the body to be identified and/or tracked, such asthe head. For such an embodiment, the cameras 140, 540 may have anenhanced resolution and/or optical zooming capability to providerecognition of miniature shapes.

Of course, the identity characteristics 1602 may include any otherproperties of the targets projected by the illuminators 220, 221.Additionally, the identity characteristics 1602 may be combinations ofthe above-described properties. For example, each illuminator 220, 221may have a different shape and a different pulsing pattern to enhancethe reliability of the differentiation process.

The database 1600 may also include a plurality of identities 1604, whichare simply labels for the objects to be identified. Each of theidentities 1604 may correspond to one of the identity characteristics1602. In FIG. 16, it is assumed that the objects to be identified arepeople. Thus, the identities 1604 are names of people. Of course, theidentities 1604 need not be names, but may be any set of uniquedesignations such as numbers or the like. However, names may be helpfulfor purposes of user interfacing with the database 1600.

“Bob” may be the father in a household, with “Bertha” as the mother.“Helga” may be a young daughter, while “Duane” is a guest. Each of theidentities 1604 may have one or more settings 1606 that define actionsto be taken by the system 1500 in response to detection of the identity1604.

For example, the settings 1606 may include an access level 1622 thatdefines the degree to which each of the identities 1604 is able toaccess the system 1500. Bob and Bertha, as the parents, have “full”access, and are thus able to make conference calls, operate thethermostat 550 and security control panel 560, and perform any otherfunctions controlled through the system 1500. As a guest, Duane has noindependent access. Helga has limited access, for television viewing andlocal calls only. The access level 1622 may thus prevent Duane and Helgafrom making long distance calls unless Bob or Bertha is present.

The settings 1606 may also include a status level 1624 that defines thedegree of priority each identity 1604 possesses with respect to systemoperation. More specifically, the status level 1624 may specify whataction should be taken in the event that two identities 1604 withconflicting settings are simultaneously present. Dominant status levels1624 may generally receive priority. In the event that avideoconferencing call is in progress, however, the call may takeprecedence. The operation of the status level 1624 will be described ingreater detail subsequently.

Additionally, the settings 1606 may include an entertainment preference1626 that defines entertainment functions that are to be automaticallyinitiated upon detection of the associated identity 1604. For example,Bob may prefer that the television automatically be turned on to a newschannel when he is present. Helga may prefer cartoons, while Duane likesclassical music and Bertha favors heavy metal. If Bertha and Duane aresimultaneously present, the system 1500 may revert to Bertha'spreference because her status level 1624 is “dominant.”

The settings 1606 may also include an environment preference such as athermostat preference 1628. The thermostat preference 1628 defines thepreferred setting of the thermostat 550 for each identity 1604. Otherenvironment preferences, such as fan operation, humidifier operation,and even door or window operation, may also be included within thesettings 1606, if desired.

Furthermore, the settings 1606 may include preferences related tooperation of the cameras 140, 540 for videoconferencing. For example, atracking preference 1630 may specify under what conditions the identity1604 is to be tracked during video communication. Some people may wish,for example, to be continuously visible while they communicate, whileothers would prefer to escape the camera's field-of-view. Similarly, azooming preference 1632 may determine the degree of zooming that iscarried out during video communication.

The settings 1606 described previously are only examples; many otherparameters of the user's environment may be altered automatically by thesystem 1500. Identity detection may be performed substantially with thesame hardware used to perform tracking; however, additional software orfirmware modules may be added to perform identity storage,identification, and system control. These modules will be shown anddescribed in greater detail in FIG. 17.

Referring to FIG. 17, a logical block diagram 1700 shows one possiblemanner in which light and signals may interact in the videoconferencingsystem 1500 of FIG. 15. FIG. 17 depicts substantially the same view asFIG. 9; however, the tracking subsystem 950, the zoom subsystem 952, andthe video preparation subsystem 954 have been collapsed for clarity. Astorage subsystem 1710, an identification subsystem 1712, and a settingcontrol subsystem 1714 have been added.

As with FIG. 9, the illustrated steps/components of FIG. 17 may beimplemented in hardware, software, or firmware using any of thecomponents of FIG. 8, alone or in combination. While various componentsare illustrated as being disposed within a STB 502, those skilled in theart will recognize that similar components may be included within thecamera 140, itself.

The operation of the emitter 130, object 212, illuminator 220, ambientlight sources 930, and camera 140 may be substantially as described inconnection with FIG. 9. Hence, the STB 502 once again receives thevisible light component 940 and the invisible light component 942. Inthe context of a videoconferencing system 1500 with multiple cameras,the STB 502 may receive visible and invisible light components 940, 942from multiple cameras 140, 540 and process them either simultaneously orin rapid sequence.

Similarly, the tracking subsystem 950, the zoom subsystem 952, and thevideo preparation subsystem 954 may operate substantially as describedpreviously. The invisible component 942 is routed to the trackingsubsystem 950 to enable tracking. The zoom subsystem 952 may interactwith the object 212 to permit determination of the range of the object212 from the camera 140.

The storage subsystem 1710 may obtain and store the data that forms thedatabase 1600. Hence, the storage subsystem 1710 may have a databaseinterface 1720 that obtains the identity characteristics 1602, theidentities 1604, and the settings 1606. The database interface 1720 mayutilize a menu system displayed on the TV 504, into which the user canenter the data or select from a plurality of options. Alternatively, thedatabase interface 1720 may be designed to receive the data in the formof vocal commands.

If desired, the database interface 1720 may track user activity andintelligently determine or suggest the settings 1606 based on patternsof activity. For instance, the system may notice when an identifiedindividual changes temperature settings within a room, and may storecorresponding settings within the database 1600. In any case, theidentity characteristics 1602, identities 1604, and identity settings1606 may then be stored within the database 1600 for subsequentretrieval.

The identification subsystem 1712 may process the invisible lightcomponent 942 to determine the identities 1604 associated with allidentity characteristics 1602 present in the invisible light component942, i.e., all objects with illuminators 220, 221 attached within thefield-of-view 160 of the camera 140. The identification subsystem 1712may also determine which illuminators 220, 221 are present within thefields-of-view of other cameras connected to the STB 502, such as thesecond camera 540.

The identification subsystem 1712 may have an identity characteristicextraction module 1730 that receives the invisible light component 942and analyzes it to determine which identity characteristics 1602 arepresent. Thus, the identity characteristic extraction module 1730 mayincorporate the pattern recognition algorithms or other algorithmsdescribed previously. The identity characteristic extraction module 1730may thus scan the invisible light component 942 and submit alldiscovered identity characteristics 1602 to a recognition engine 1732.

The recognition engine 1732 may receive the identity characteristics1602 and reference the database 1600 to determine which identities 1604correspond to each of the identity characteristics 1602. The identities1604 may then be conveyed to the setting control subsystem 1714. Thesetting control subsystem 1714 may utilize the identities 1604 todetermine how to control other subsystems and devices.

More specifically, the setting control subsystem 1714 may have a settingretrieval module 1740 that receives the identities 1604 from therecognition engine 1732. The setting retrieval module 1740 referencesthe database 1600 to obtain all relevant settings 1606 for each of theidentities 1604. The settings 1606 may be conveyed to a command module1742. The command module 1742 may utilize the settings 1606 to determinewhich operational commands 1744 are to be conveyed to which subsystemsor devices.

For example, operational commands 1744 may be delivered to the trackingsubsystem 950 and the zoom subsystem 952 based on the tracking andzooming preferences 1630, 1632. Operational commands 1744 may also betransmitted to external devices such as the TV 504, thermostat 550,security control panel 560, and any other devices controlled by the STB502.

If desired, feedback systems (not shown) may also be present between theexternal devices 504, 550, 560, etc. and the STB 502 so that the commandmodule 1742 can learn from behavior patterns of users and/or take thecurrent status of the external devices 504, 550, 560 into account. Forexample, a person (i.e., an identity 1604) may have no establishedentertainment preference 1626, but may consistently turn on the TV 504to watch basketball games. The command module 1742 may sense the patternof activity and change the entertainment preference 1626 thatcorresponds to the identity 1604 accordingly, so that the TV 504 isautomatically turned on and switched to a basketball game (if available)when the person enters the area.

Similarly, a person may have a thermostat preference 1628 of 65°. If theperson has just manually changed the thermostat 550 to 70°, left thearea, and returned, the command module 1742 may temporarily override thethermostat preference 1628 and leave the thermostat 550 at 70°.

The video preparation subsystem 954 may determine which camera 140, 540is to be active. More specifically, if the STB 502 is connected tomultiple cameras 140, 540, as shown in FIG. 15, the STB 502 may activatethe camera 140 or 540 with the best view of the communicating person. Inthe course of operation, the tracking subsystem 950 obtains anapproximate location of the object to be tracked, as describedpreviously.

Such positional information may be utilized by the video preparationsubsystem 954 to determine which camera 140, 540 is in the best positionfor viewing the communicating person. The video preparation subsystem954 may process and transmit the visible light component 940 of thecamera 140 or 540 that is in the best position.

Of course, the various subsystems 950, 952, 954, 1710, 1712, 1714 of theSTB 502 may operate cyclically. Thus, the system 1500 may becontinuously checking to see if any new identities 1604 have arrivedwithin the fields-of-view 160, 1560, and issuing operational commands1744 when new identities 1604 are detected. Similarly, the videopreparation subsystem 954 may continuously monitor the location of thecommunicating identity 1604, and may change active cameras 140, 540 assoon the communicating identity 1604 moves from one field-of-view 160,1560 to the other. One possible mode of operation of the system 1500will be shown and described in greater detail in conjunction with FIG.18.

Referring to FIG. 18, one embodiment of an identification and controlmethod 1800 is depicted. The method 1800 may be carried outperiodically, such as once every second, or even once for every framereceived by the cameras 140, 540. Furthermore, the method 1800 may becarried out independently for each of the cameras 140, 540, since eachcamera 140, 540 is positioned in a different area, in which differentcommands are to be carried out.

Initially, the STB 502 may receive 1810 the invisible light component942 from the camera 140 or 540. All discernable identity characteristics1602 may then be extracted 1812 from the invisible light component 942and compared 1814 with the database 1600 to obtain the identities 1604associated with each of the identity characteristics 1602.

Once the identities 1604 have been obtained, the system 1500 maydetermine 1820 whether only one identity 1604 is present. If only oneidentity 1604 is present, the present identity 1604 is set 1822 to bethe dominant identity 1604, regardless of the status level 1624 of theidentity 1604.

If there is not only one identity 1604 present, the system 1500 maydetermine 1830 whether multiple identities 1604 are present. If so, thesystem 1500 may proceed to determine 1832 which of the detectedidentities 1604 is dominant. Such a determination may be made, forexample, by comparing the status levels 1624 of all identities 1604present. If multiple identities 1604 have the same status level 1624,for example, two identities 1604 are “dominant,” the system 1500 mayselect one based upon usage history, proximity, random selection, or anyother method.

If a single identity 1604 is not present and multiple identities 1604are not present, then no identities 1604 are present. The system 1500may then determine 1840 whether no identities 1604 have been present fora threshold time period. If no identities 1604 have been present for thethreshold time, the system 1500 may perform 1842 functions according todefault settings. For example, the default settings may be to turn offall entertainment devices and lights, activate the security controlpanel 560, set the thermostat 550 to an energy efficient level, etc.

Assuming one or more identities 1604 are present, the system 1500 maydetermine 1850 the settings 1606 for all identities 1604 present. Ofcourse, selected settings 1606, such as the access level 1622 and statuslevel 1624, may have already been determined in conjunction withprevious steps. The system 1500 may then determine 1852 whether a videocall is in progress for one of the identities 1604 present. The callingstatus of an identity 1604 may be listed in the status level 1624 fieldof the database 1600 because a video call may give the caller precedenceover settings 1606 of other identities 1604.

If a video call is in progress, the system 1500 may determine 1854whether the camera 140 or 540 that received 1810 the invisible component942 is the best camera 140 or 540 for viewing the identity 1604 engagedin the video call. As mentioned previously, this determination may bemade with the aid of the tracking subsystem 950; the zoom subsystem 952may also provide useful positional data for the identity 1604 incommunication. The determination of which camera 140, 540 is best toview the identity 1604 may depend upon the range from the identity 1604to each camera 140, 540, the direction in which the identity 1604 ismoving, the position of the identity 1604 within the fields-of-view 160,1560 of the cameras 140, 540, and other factors.

If the camera 140 or 540 is the best for viewing the identity 1604 invideo communication, the system 1500 may activate 1860 the camera 140 or540. In this context, “activation” refers to usage of the camera 140 or540 to receive and transmit the visible light component 940 to a remoteterminal (not shown) for display. In order to conduct identification,the STB 502 may continuously receive and process the invisible lightcomponents 942 from all cameras 140, 540, and possibly even the visiblelight components 940. The visible light component 940 from a givencamera 140 or 540 simply is not transmitted to another party until thecamera 140 or 540 is activated.

Once the camera 140 or 540 has been activated, the system 1500 may trackand/or zoom in on the communicating identity 1604 through any of themethods described previously. Accordingly, tracking and/or zooming 1862may entail moving the camera 140 or 540, or cropping the visiblecomponent 940 to carry out software steerable tracking and zooming. Asmentioned previously, tracking and zooming 1862 may be carried outaccording to the tracking and zooming preferences 1630, 1632 for thecommunicating identity 1604.

Regardless of whether the camera 140 or 540 is the best for viewing thecommunicating identity 1604, the system 1500 may perform 1870 otherfunctions according to the settings 1606 for the dominant identity 1604,and possibly other identities 1604. Functions that utilize audible orvisual media may be overridden by the video call; hence, thecommunicating identity 1604 may receive priority over entertainmentpreferences 1626 and the like. Once the call terminates, the system 1500may then revert to the entertainment preference 1626 of the dominantidentity 1604.

If desired, the status level 1624 may not operate universally. Forexample, the status level 1624 need not apply to all other settings1606. Each of the settings 1606 may even have its own status level.Thus, when both Bertha and Helga are present, Bertha's thermostatpreference 1628 may still receive priority, while Helga's entertainmentpreference 1626 takes precedence.

Furthermore, each of the settings 1606 may have its own customizedversion of the access level 1622. Hence, Helga may be permitted tochange the thermostat 550 only within the range of 60° to 80°.Similarly, Helga may be permitted to watch only television programmingwith a rating of TV-PG or better. Duane may be permitted to use thethermostat 550, but not the security control panel 560.

As mentioned previously, the method 1800 described above is repeatedfrequently for each of the cameras 140, 540. Hence, when a new identity1604 enters the area of one of the cameras 140, 540, the system 1500 mayreact rapidly if the settings 1606 require that any action be taken.

Similarly, during videoconferencing, the communicating identity 1604, orthe person 210 in the example of FIG. 15, may move from thefield-of-view 160 of the first camera 140 to the field-of-view 1560 ofthe second camera 540. In such a situation, when the method 1800 iscarried out for the first camera 140, it is determined that the firstcamera 1854 is no longer the best for viewing the person 210.

When the method 1800 is carried out for the second camera 540, thesystem 1500 determines that the second camera 540 is the best forviewing the person 210. Hence, tracking and/or zooming 1862 are carriedout with respect to the second camera 540, and the visible component 940from the second camera 540 is transmitted to the remote terminal by theSTB 502. The method 1800 may operate rapidly and frequently enough thatthe handoff from one camera to the other is substantially seamless forthe person with whom the person 210 is communicating.

As previously noted, a person 210 may wish to hide a portion of thevisible component 940 from the party with whom he or she iscommunicating. For example, the person 210 may wish to hide his or herdwelling, other people, or part of his or her body. Hence, it isdesirable to provide a method by which a portion of the visiblecomponent 940 may be selectively obscured. Such a method will be shownand described in connection with FIGS. 19 through 22.

Referring now to FIG. 19, a zoomed and centered view 1900 of visibleoutput from the camera 140 or 540 is shown. The view 1900 is similar tothe view 400 of FIG. 4A. However, FIG. 19 includes a halo 1910 with agenerally elliptical shape. As previously noted, a halo 1910 is aninvisible outline or stencil that is superimposed upon the visiblecomponent of the video signal. In one embodiment, a halo 1910 is used todivide the visible component into a portion to be obscured (typicallythe portion outside of the halo 1910) and a portion to be left unchanged(typically the portion inside of the halo 1910).

The view 1900 is thus divided into a portion 1920 inside the halo 1910and a portion 1930 outside the halo 1910. The portion 1930 may beobscured, while the portion 1920 is left intact so that only the head212 and a portion of the neck of the person 210 are visible to therecipient of the visible component 940.

Such a view may be desirable if the person 210 prefers not to show therest of his body to the other party. For example, the person 210 mayhave just awakened or stepped from the shower, and may thus not be fullyclothed. The person 210 may also be engaged in some action that he doesnot wish to reveal to the other party.

The halo 1910 may be positioned such that the center of the halo 1910 ishorizontally centered with respect to the illuminator 220. Since, asdepicted in FIG. 19, the illuminator 220 may not disposed at thevertical center of the head 212, the center of the halo 1910 need not bevertically aligned with the illuminator 220. The halo 1910 may rather bepositioned at an established vertical displacement with respect to theilluminator 220. The person 210 may be expected to turn his head 212 toeither side. The elliptical shape of the halo 1910 enables the halo 1910to move horizontally to follow the illuminator 220 without causing thehead 212 to leave the halo 1910.

In certain situations, the halo 1910 may remain substantially motionlessdue to the operation of the tracking subsystem 950 and the zoomsubsystem 952. More specifically, as described previously, the trackingsubsystem 950 may keep the field-of-view 160 centered on the illuminator220. Similarly, the zoom subsystem 952 may adjust the zoom level to keepthe illuminator 220 at a constant size with respect to the field-of-view160. If the person 210 is always at the same position and magnificationlevel within the field-of-view 160, no automatic alteration of the sizeor position of the halo 1910 is necessary.

In other situations, it may be desirable to continuously reposition andresize the halo 1910 with the aid of the tracking subsystem 950 and thezoom subsystem 952. Repositioning and resizing the halo 1910 may beparticularly desirable when tracking and/or zooming are not beingperformed, as when the person 210 has selected to conductvideoconferencing without tracking and/or zooming. Halo application maybe performed independently of tracking and zooming.

The person 210 then moves during videoconferencing without acorresponding change in the position or size of the field-of-view 160.The position and distance data from the tracking subsystem 950 and thezoom subsystem 952 may then be used to continuously reposition andresize the halo 1910 to maintain the privacy of the person 210 as hemoves within the field-of-view 160.

Several characteristics of the halo 1910 may be user selectable. Forexample, the person 210 may wish to vary the size of the halo 1910 toshow more or less of his body and/or surroundings. The person 210 mayalso wish to change the position of the halo 1910; for example, if theperson 210 wishes to show his entire shirt collar, he may wish to movethe halo 1910 downward, and possibly enlarge the halo 1910 to avoidcovering any of the head 212.

Additionally, the person 210 may wish to alter the shape of the halo1910. If the person 210 wishes to show his entire shirt collar, he maywish to increase the vertical dimension of the halo 1910 withoutsubstantially changing the horizontal dimension. Thus, the aspect ratioof the halo 1910 may be altered to form a circle or vertically elongatedellipse.

Alternatively, the overall shape of the halo 1910 may be changed. Forexample, other users of the system 1500 may prefer shapes such as adiamond, star, sun, moon, box, or the like. Animated halos may be usedto add additional character to a video communication. Each identity 1604may have its own assigned set of halo characteristics, i.e., its ownhalo type. The halo type may include not only the shape of the halo, butalso the size, position, animation, and any other desiredcharacteristics. Thus, while Bob prefers the vertically elongatedellipse sized slightly larger than his head, Helga may prefer a spinningstar large enough to show her entire upper body.

If desired, each of the identities 1604 may have multiple halo typesthat are applied according to which camera 140, 540 is viewing theidentity 1604. For example, if the camera 140 is positioned in a familyroom or other comparatively public room, the person 210 may have acomparatively large halo 1910 or no halo at all. If the camera 540 ispositioned in a bedroom, bathroom, or other comparatively private place,the camera 540 may provide a smaller halo 1910 as shown in FIG. 19.

As mentioned previously, the system 1500 may automatically activate thecamera 140 or 540 that is best for viewing the person 210.Simultaneously, the system 1500 may automatically apply thecorresponding halo type. Thus, when the person 210 moves from thefield-of-view 160 of the first camera 140 to the field-of-view 1560 ofthe second camera 540, the smaller halo 1910 may be automaticallyapplied to provide greater privacy.

In certain circumstances, it may be desirable to have a halo shaped tomore closely follow the outline of a person's body. The person can thenutilize hand gestures and other motions as part of the communication.Such an embodiment is depicted in FIG. 20.

Referring to FIG. 20, a view 2000 of visible output from the camera 140or 540 is shown. The view 2000 is similar to the view 300 of FIG. 3,with the addition of a halo 2010 shaped to generally resemble a portionof a human body. The halo 2010 may have a head portion 2012, an upperbody portion 2014, and a lower body portion 2016.

The halo 2010 may have a shape that generally outlines a person directlyfacing the camera 140 or 540. The head portion 2012 may be somewhatwider than the head 212, like the halo 1910 of the previous figure, toensure that the head 212 remains visible. The upper body portion 2014may be generally flat-sided so that the arms of the person 210 will bevisible when they are close to the torso, but not when they areextended. The lower body portion 2016 is somewhat narrower than theupper body portion 2014, and is truncated by the bottom edge of thefield-of-view 160.

As shown, the illuminator 220 may be disposed on the torso of the person210 instead of on the head 212. The illuminator 220 may, for example, beattached to a shirt collar, necklace, or the fabric of the shirt in amanner similar to a name tag. Thus, the person 210 may move his head 212without significantly altering the horizontal or vertical position ofthe halo 2010. In FIG. 20, the halo 2010 is horizontally centered on theilluminator 220.

Since the halo 2010 is wide enough to accommodate the person 210 whenfacing the camera 140 or 540, the halo 2010 is somewhat wider than theperson 210 when the body of the person 210 is angled from the camera 140or 540. Hence, a small portion of the background behind the person 210may be visible within the halo 2010 when the person 210 is not directlyfacing the camera 140 or 540, as in the view of FIG. 20.

If desired, the aspect ratio of the halo 2010 may be automaticallyadjusted based upon the orientation of the illuminator 220. The aspectratio of the target of invisible light received from the illuminator 220indicates the orientation of the illuminator 220 with respect to thecamera 140 or 540.

The aspect ratio of the target is the width of the target divided by theheight of the target. If the illuminator 220 directly faces the camera140 or 540, the target will have the same aspect ratio as theilluminator 220. However, if the illuminator 220 is tilted upward ordownward, the aspect ratio will be increased. Conversely, if theilluminator is tilted to the left or right, the aspect ratio will bedecreased.

The system 1500 may assume that the illuminator 220 lies flat againstthe body of the person 210, and that tilting of the illuminator 220corresponds with tilting of the body of the person 210. Thus, the system1500 may keep the aspect ratio of the halo 2010 proportional to theaspect ratio of the target reflected by the illuminator 220. Hence, whenthe person 210 hunches downward or reclines in a chair, the halo 2010may automatically become vertically shorter. Similarly, when the person210 turns to one side or the other, the halo 2010 may automaticallybecome horizontally narrower.

The halo 2010 may also have multiple orientations that can be selectedby the person 210. For example, the person 210 may selectively disposethe halo 2010 in a horizontal orientation so that the person 210 can liehorizontally within the field-of-view 160 to relax or exercise duringvideoconferencing.

In certain embodiments, the illuminator 220 may be omitted entirely infavor of halo sizing and positioning based on the infrared signaturegenerated by the body of the person 210. As mentioned previously, insome configurations, tracking and zooming may be performed withreference to the body's infrared signature. Halo positioning and sizingmay be carried out in substantially the same manner. If desired, thehalo 2010 may simply be positioned to outline the area of intensityproduced by the infrared radiation of the person. Thus, the halo 2010may more nearly conform to the outline of the body of the person 210, orjust the head 212.

The halos 1910, 2010 of FIGS. 19 and 20 are only examples of possiblehalos. As mentioned previously, halos according to the invention may beshaped, positioned, sized, and generally applied in a wide variety ofways. Halo application may be performed substantially with the samehardware used to perform tracking, zooming, and identification. However,additional software or firmware modules may be added to perform haloselection, halo positioning, halo sizing, signal modification, and anyother desirable function related to signal obscuring. Exemplary modulesfor performing these functions will be shown and described in greaterdetail in FIG. 21.

Referring to FIG. 21, a logical block diagram 2100 shows one possiblemanner in which light and signals may interact in the videoconferencingsystem 1500 of FIG. 15 to enable selective obscuring of the visiblecomponent 940. FIG. 21 depicts substantially the same view as FIGS. 9and 17. However, the tracking subsystem 950, the zoom subsystem 952, thevideo preparation subsystem 954, the storage subsystem 1710, theidentification subsystem 1712, and the setting control subsystem 1714have been collapsed for clarity. An obscuring subsystem 2110 has beenadded to provide selective obscuring of the visible component 940.

As with FIGS. 9 and 17, the illustrated steps/modules/components of FIG.21 may be implemented in hardware, software, or firmware using any ofthe components of FIG. 8, alone or in combination. While variouscomponents are illustrated as being disposed within a STB 502, thoseskilled in the art will recognize that similar components may beincluded within the camera 140, itself.

The operation of the emitter 130, object 212, illuminator 220, ambientlight sources 930, and camera 140 may be substantially as described inconnection with FIG. 9. Hence, the STB 502 once again receives thevisible light component 940 and the invisible light component 942. Inthe context of a videoconferencing system 1500 with multiple cameras,the STB 502 may receive visible and invisible light components 940, 942from multiple cameras 140, 540 and process them either simultaneously orin rapid sequence.

Similarly, the tracking subsystem 950, the zoom subsystem 952, and thevideo preparation subsystem 954 may operate substantially as describedabove. The invisible component 942 is routed to the tracking subsystem950 to enable tracking. The zoom subsystem 952 may interact with theobject 212 to permit determination of the range of the object 212 fromthe camera 140.

The identification subsystem 1712 may also receive the invisiblecomponent 942 to enable retrieval of identities 1604 from the storagesubsystem 1710. The setting control subsystem 1714 may receive theidentities 1604 to generate operational commands 1744 based on settings1606 stored by the storage subsystem 1710.

The obscuring subsystem 2110 may process the visible component 940 toobscure the portion that the person 210 does not wish to bedistinguishable to the other party. More particularly, the obscuringsubsystem 2110 may selectively apply a halo 1910, outside of which thevisible component 940 is obscured. For the remainder of the discussion,the use of the halo 1910 of FIG. 19 will be assumed.

Thus, the obscuring subsystem 2110 may have a halo selection module 2120that selects the type of halo 1910, or halo type 2122, to apply. Ifdesired, the halo selection module 2120 may receive operational commands1744 from the setting control subsystem 1714 that determine what halotype 2122 is selected. As mentioned previously, the halo type 2122 mayinclude the halo size, halo position, halo shape, and any othercustomizable characteristics. Such characteristics may be stored assettings 1606 within the database 1600.

The database 1600 may have default halo types 2122. Alternatively, thehalo type 2122 may be user-selected through the use of menus displayedon the television 504, verbal commands, or the like. Of course, the useof multiple halo types 2122 is optional; if desired, the halo selectionmodule 2120 may be omitted and one halo type 2122 may simply be appliedfor every communication.

After the identification subsystem 1712 identifies all of the identities1604 that are present, the setting control subsystem 1714 may determinewhich, if any, is currently engaged in videoconferencing. The settingcontrol subsystem 1714 may retrieve settings 1606 for the communicatingidentity 1604, including settings 1606 related to the halo type 2122.The settings 1606 may be converted into operational commands 1744 andconveyed to the halo selection module 2120. The halo selection module2120 may process the operational commands 1744 to establish the halotype 2122 for the current communication.

The obscuring subsystem 2110 may also have a halo sizing module 2130that establishes a halo size 2132. Although the size of the halo 1910with respect to the person 210 may be specified by the halo type 2122,the halo 1910 may be dynamically enlarged or reduced as the person 210approaches or retreats from the camera 140 or 540. As mentionedpreviously, the zoom subsystem 952 may, in some configurations, obviatethe need for dynamic halo resizing. Hence, like the halo selectionmodule 2120, the halo sizing module 2130 is optional.

As described previously, the zoom subsystem 952 may measure or calculatethe distance 972 between the object, i.e., the person 210, and thecamera 140 or 540. The halo sizing module 2130 may cyclically receivethe distance 972 and utilize the distance 972 to determine the size ofthe halo 1910. More precisely, the halo sizing module 2130 maycontinuously adjust the size of the halo 1910 to maintain a relativelyconstant ratio between the visible size of the person 210 and thevisible size of the halo 1910. Thus, the halo 1910 constantly has thesame size with respect to the person 210, regardless of the distance972.

Similarly, the obscuring subsystem 2110 may have a halo positioningmodule 2140 that establishes a halo position 2142. The halo position2142 may also be specified by the halo type 2122, but may be dynamicallychanged to maintain proper positioning with respect to the person 210while the person 210 moves. Dynamic halo positioning may, in somesituations, be obviated by the tracking subsystem 950. Hence, like thehalo selection module 2120 and the halo sizing module 2130, the halopositioning module 2140 is optional.

As described previously, the tracking subsystem 950 may process theinvisible component 942 to determine an object vector 150 from thecamera 140 or 540 to the person 210. The object vector 150 may becyclically received by the halo positioning module 2140 to determine theproper position for the halo 1910. The halo positioning module 2140 maycontinuously adjust the position of the halo 1910 to keep the halo 1910properly positioned with respect to the person 210 while the person 210moves within the field-of-view 160 or 1560.

The obscuring subsystem 2110 may further have a signal modificationmodule 2150 that receives the halo type 2122, the halo size 2132, andthe halo position 2142. The signal modification module 2150 may receivethe visible component 940 and obscure the portion of the visiblecomponent 940 that lies outside the halo 1910, e.g., the portion 1930 ofFIG. 19.

“Obscuring” refers to alteration of the portion 1930 in some manner thatmakes objects within the portion 1930 more difficult for the receivingparty to see or distinguish. The signal modification module 2150 mayobscure the portion 1930 in a variety of ways. The signal modificationmodule 2150 may thus provide an obscured visible component 2160.

As one example, the signal modification module 2150 may blur the portion1930. Blurring may be accomplished in a variety of ways known in the artof image processing. For example, pixels of the portion 1930 may begrouped and re-mapped within each group so that edges and smallerfeatures are indistinguishable. As another example, blocks of pixels maybe grouped and colored alike to provide a blocky appearance. Of course,many other blurring methods may be used in the alternative to or incombination with the above.

As an alternative, the signal modification module 2150 may obscure theportion 1930 by changing the pixels of the portion 1930 to a uniformcolor, or by re-coloring the pixels toward a uniform color. For example,the portion 1930 may simply be changed to a black or gray color. Asanother example, the pixels of the portion 1930 may be changed toward agray color by decreasing the saturation level of each pixel by anestablished amount. The saturation level of each pixel may be decreasedby an established amount to move the portion 1930 toward a black color,in which shapes are less distinguishable, although the color is notuniform.

As yet another alternative, the signal modification module 2150 mayreplace the portion 1930 with an unrelated image. For example, thesignal modification module 2150 may store images of scenic backgroundssuch as a beach, a forest, a high-rise office window, and the like. Thedesired image can then be selected by the person 210 and the image or apointer to the image can be stored as one of the settings 1606 forretrieval in the form of an operational command 1744. The image mayalternatively be an advertisement or other message provided by thenetwork 501.

Of course, numerous other obscuring methods are known in the art. In anycase, the signal modification module 2150 alters the visible lightcomponent 940 such that the portion 1930 is obscured to some degree.

The signal modification module 2150 may provide a sharp transitionbetween the portion 1920 inside the halo 1910 and the portion 1930outside the halo 1910. For example, the portion 1920 may be entirelyclear, while the portion 1930 is obscured with a consistent level ofalteration. Thus, the edges of the halo 1910 are distinct within theobscured visible component 2160.

Alternatively, the signal modification module 2150 may graduallyincrease the level of alteration of the visible component 940 across theedge of the halo 1910. Thus, the halo 1910 may appear to have anindistinct or gradual edge. Such an effect may be desirable to give thehalo 1910 a softer, less harsh appearance.

If desired, the portions 1920, 1930 may even be swapped such that theportion 1920 inside the halo 1910 is obscured while the portion 1930outside the halo 1910 is left unchanged. Obscuring material inside thehalo 1910 may be desirable in certain circumstances, such as when theperson 210 wishes to hide his identity by concealing his face 212, orwhen the person 210 wishes to conceal an activity he is performing withhis hands while leaving the remainder of field-of-view 160 unchanged.

Once the signal modification module 2150 has obscured the portion 1920or 1930, the obscured visible component 2160 may be conveyed to thevideo preparation subsystem 954. The video preparation subsystem 954 mayfurther format and/or process the obscured visible component 2160 in themanner illustrated in FIG. 9. Hence, the obscured visible component 2160may be cropped and otherwise processed by the video preparationsubsystem 954 to carry out the instructions provided by the trackingsubsystem 950 and the zoom subsystem 952. If mechanical tracking andzooming are used, the obscured visible component 2160 may be ready fortransmission without further processing.

Referring to FIG. 22, one embodiment of a halo application method 2200is shown. The halo application method 2200 may be used with the logicalcomponents depicted in FIG. 21. Since halo application may be integratedwith tracking and zooming, the method 2200 may include steps frompreviously described methods.

Initially, the system 1500 may use the camera 140 or 540 to capture 1040a frame of a first video signal, including the visible component 940 andthe invisible component 942. As with the method 1000, the method 2200may be carried out on a frame-by-frame basis. Hence, capturing 1040 mayentail the receipt of a single video frame.

The system 1500 may then determine 2210 whether a new object is toreceive a halo 1910. For example, an object that did not have a halo1910 in a previous frame may be selected to receive a halo 1910, eitherby a user or by the logic of the system 1500. The system 1500 may, forexample, detect the entry of a new object into the field-of-view 160 or1560 and automatically apply a halo 1910 to the object. The system 1500may simultaneously apply multiple halos 1910 to multiple objects.Alternatively, the system 1500 may only apply a single halo 1910 at atime, and may determine which object is visible within the halo 1910,for example, by referring to the status level 1624 of the correspondingidentity 1604.

If a new object is to receive a halo 1910, the system 1500 may determine2220 the identity 1604 of the new object, for example, through the useof the method 1800. The system 1500 may then reference 2230 the settings1606 for the identity 1604 to determine what type of halo 1910 to apply.

Once the steps 2220 and 2230 have been carried out, or if no new objectis to receive a halo 1910, the system 1500 may then process theinvisible component 942 to determine 1110 the location of the targetwithin the field-of-view 160, 1560. Determining 1110 the location of thetarget within the field-of-view 160, 1560 may be performed insubstantially the same way as described previously in connection withFIG. 11. The system 1500 may then calculate 1120 the object vector 150from the camera 140 or 540 to the illuminator 220, or to a correspondingtarget of body heat. Once the object vector 150 is known, the system1500 may establish 2250 a halo position 2142 with respect to the objectvector 150.

The system 1500 may then determine 1310 the distance 972 between theobject and the camera 140, 540, in much the same manner as described inconjunction with FIG. 13. As mentioned previously, determination of thedistance 972 may be carried out through hardware, software, or anycombination thereof. Once the distance 972 is known, the halo size 2132may be established 2270 based on the distance 972. If desired, the halosize 2132 may simply be kept proportional to the distance 972. Hence,when the object, e.g. the person 210, moves nearer the camera 140 or1540, the halo 1910 may expand proportionally, and vice versa.

After the halo size 2132 and the halo position 2142 have beenestablished 2250, 2270, the system 1500 may apply 2280 the halo 1910 tothe visible component 940 by obscuring the portion 1930 that liesoutside the halo 1910. As mentioned previously, the portion 1920 thatlies inside the halo 1910 may alternatively be obscured. Furthermore,obscuring may be performed using any of the methods describedpreviously, or any combination thereof.

Once obscuring has been performed, the system 1500 may determine 2290whether the halo 1910 is to be applied on a continuing basis.Determining 2290 whether the halo 1910 is to be applied may entaildetermining whether a user has submitted a command to stop transmittingthe visible component 940, or to simply stop applying the halo 1910. Ifapplication of the halo 1910 is to continue, the method 2200 may onceagain capture 1040 the video signal. If application of the halo 1910 isto be terminated, the method 2200 ends accordingly. Of course, themethod 2200 is restarted when a user or the system 1500 determines thatthe halo 1910 is to be applied again.

Based on the foregoing, the present invention offers a number ofadvantages not available in conventional approaches. Objects, includingpeople, may be automatically tracked, zoomed, and identified when theyenter a field-of-view of a camera. The identification can be used toensure that the correct object is tracked during videoconferencing withthe proper tracking and zooming parameters. The object may be trackedseamlessly during movement from one camera to another.

Furthermore, the visible component of the video signal may be partiallyobscured to hide the surroundings of the object that is the subject ofthe video conference. A user may select the shape, size, and position ofthe halo to be used. The halo may automatically follow the object duringmotion of the object, and may be resized automatically as the objectapproaches or moves away from the camera. Hence, the user can have arelatively high degree of control over how much of the scene isdiscernable to the other party. Through the use of the presentinvention, the above described benefits can be obtained withoutexcessively expensive or inconvenient equipment.

While specific embodiments and applications of the present inventionhave been illustrated and described, it is to be understood that theinvention is not limited to the precise configuration and componentsdisclosed herein. Various modifications, changes, and variationsapparent to those skilled in the art may be made in the arrangement,operation, and details of the methods and systems of the presentinvention disclosed herein without departing from the spirit and scopeof the invention.

What is claimed is:
 1. A system for automatically obscuring a portion ofa video signal, the system comprising: a camera, sensitive to invisiblelight, that captures a video signal depicting an object, the videosignal having visible and invisible components; a tracking subsystemthat processes the invisible component to determine a position of aspecific target of invisible light projected from the object, whereinthe target includes at least one characteristic that distinguishes itfrom other targets; a halo positioning module that automaticallypositions a halo in the visible component with respect to the positionof the target; and a signal modification module that obscures a portionof the visible component that lies outside the halo.
 2. The system ofclaim 1, further comprising an illuminator disposed on the object toproject the target of invisible light.
 3. The system of claim 2, whereinthe illuminator comprises a reflector, the system further comprising: astationary invisible light emitter that generates invisible lightreflected by the reflector to project the target.
 4. The system of claim2, further comprising a portable invisible light emitter that generatesinvisible light to project the target.
 5. The system of claim 1, furthercomprising an identification module to identify the object based on theat least one characteristic of the target.
 6. The system of claim 2,wherein the illuminator is attached to an article worn by a person,wherein the article is selected from the group consisting of a pair ofglasses, a tie clip, and a piece of jewelry.
 7. The system of claim 2,wherein the illuminator comprises a coating applied directly to skin ofa person, wherein the coating projects invisible light.
 8. The system ofclaim 1, wherein the object is selected from the group consisting of aremote control device and a set of keys.
 9. The system of claim 1,wherein the camera comprises: a wide frequency charge-coupled device(CCD) that generates the visible component and the invisible componentof the video signal.
 10. The system of claim 1, wherein the cameracomprises: a first charge-coupled device (CCD) that generates thevisible component of the video signal; and a second CCD that generatesthe invisible component of the video signal.
 11. The system of claim 1,wherein the halo has a shape selected from the group consisting of acircle, an ellipse, and a polygon.
 12. The system of claim 11, whereinthe object comprises a person engaged in video communication using thecamera, the system further comprising: an illuminator disposed on a headof the person to project the target of invisible light.
 13. The systemof claim 12, wherein the positioning module horizontally positions thehalo such that a center of the halo is horizontally aligned with theilluminator.
 14. The system of claim 1, wherein the object comprises aperson engaged in video communication using the camera and the halo hasa shape that resembles at least a potion of an outline of the person.15. The system of claim 14, further comprising: an illuminator disposedon a torso of the person to project the target of invisible light. 16.The system of claim 14, wherein the invisible light comprises infraredradiation generated by the person.
 17. The system of claim 16, furthercomprising: a reflector disposed on the object to project the target ofinvisible light; a stationary invisible light emitter that generatesinvisible light reflected by the reflector to project the target,wherein the stationary invisible light emitter generates the invisiblelight at periodic intervals such that the target does not appear inevery frame of the video signal, and wherein the tracking subsystemdistinguishes a target from other sources of invisible light bydetermining that the target does not appear in every frame of the videosignal.
 18. The system of claim 1, wherein the halo positioning modulecontinuously repositions the halo in response to motion of the targetwithin the invisible component.
 19. The system of claim 1, furthercomprising: a halo sizing module that automatically establishes a sizeof the halo based on a distance of the object from the camera.
 20. Thesystem of claim 1, wherein the signal modification module graduallyincreases a level of alteration of the visible component across an edgeof the halo.
 21. The system of claim 1, wherein the signal modificationmodule obscures the portion of the visible component by replacing theportion with a single color.
 22. The system of claim 1, wherein thesignal modification module obscures the portion of the visible componentby blurring the portion to make objects within the visible componentless distinguishable.
 23. The system of claim 1, wherein the signalmodification module obscures the portion of the visible component byreplacing the portion with an image unrelated to the visible component.24. The system of claim 1, further comprising: a local display deviceviewable from within a field-of-view of the camera, the local displaydevice displaying at least a subset of the visible component of thevideo signal.
 25. The system of claim 1, further comprising: acommunication subsystem that transmits at least a subset of the visiblecomponent of the video signal to a remote terminal for display.
 26. Thesystem of claim 25, wherein the communication subsystem is configured touse a network selected from the group consisting of a cable televisionnetwork and a direct broadcast satellite network.
 27. The system ofclaim 26, further comprising: a codec that receives televisionprogramming from the communication subsystem for display on a localdisplay device viewable from within the first field-of-view, wherein thecodec and the tracking subsystem are disposed within a common housing toform a set top box, and wherein the set top box transmits the firstsignal from the camera to the communication subsystem.
 28. The system ofclaim 25, wherein the communication subsystem receives a second videosignal from the remote terminal, the system further comprising: a localdisplay device that displays the second video signal.
 29. The system ofclaim 28, wherein the second video signal and at least a subset of thevisible component of the video signal arc displayed simultaneously. 30.A system for automatically obscuring a portion of a video signal, thesystem comprising: a camera, sensitive to invisible light, that capturesa video signal depicting an object, the video signal having visible andinvisible components; a tracking subsystem that processes the invisiblecomponent to determine a position of a specific target of invisiblelight projected from the object, wherein the target includes at leastone characteristic that distinguishes it from other targets; a halopositioning module that automatically positions a halo in the visiblecomponent with respect to the position of the target; a signalmodification module that obscures a portion of the visible componentthat lies outside the halo; a halo selection module; a database thatcontains identities and settings for a plurality of objects; anidentification subsystem that receives an identity of the object fromthe database; and a setting control subsystem that retrieves theuser-selected shape that corresponds to the identity of the object andconveys the user-selected shape to the halo selection module.
 31. Asystem for automatically obscuring a portion of a video signal depictingan object, the video signal having visible and invisible components, thesystem comprising: a tracking subsystem that processes the invisiblecomponent to determine a position of a specific target of invisiblelight projected from the object, wherein the target includes at leastone characteristic that distinguishes it from other targets; a halopositioning module that automatically positions a halo in the visiblecomponent with respect to the position of the target; and a signalmodification module that obscures a portion of the visible componentthat lies outside the halo.
 32. An obscuring subsystem for automaticallyobscuring a portion of a video signal depicting an object, the videosignal having visible and invisible components, the obscuring subsystemcomprising: a halo positioning module that receives a position of aspecific target of invisible light projected from the object, whereinthe target includes at least one characteristic that distinguishes itfrom other targets, and automatically positions a halo in the visiblecomponent with respect to the position of the target; and a signalmodification module that obscures a portion of the visible componentthat lies outside the halo.
 33. A method for automatically obscuring aportion of a video signal, the method comprising: capturing a videosignal depicting an object with a camera sensitive to invisible light,the video signal having visible and invisible components; processing theinvisible component to determine a position of a specific target ofinvisible light projected from the object, wherein the target includesat least one characteristic that distinguishes it from other targets;automatically positioning a halo in the visible component with respectto the position of the target; and obscuring a portion of the visiblecomponent that lies outside the halo.
 34. The method of claim 33,further comprising disposing an illuminator on the object to project thetarget of invisible light.
 35. The method of claim 34, wherein theilluminator comprises a reflector, the method further comprising:generating invisible light with a stationary invisible light emitter;and reflecting the invisible light with the reflector to project thetarget.
 36. The method of claim 34, wherein the illuminator comprises aportable invisible light emitter, the method further comprising:generating invisible light with the portable invisible light emitter toproject the target.
 37. The method of claim 33, further comprisingidentifying the object based on the at least one character of thetarget.
 38. The method of claim 34, further comprising: attaching theilluminator to an article worn by a person, wherein the article isselected from the group consisting of a pair of glasses, a tie clip, anda piece of jewelry.
 39. The method of claim 34, further comprising:applying the illuminator directly to skin of a person in the form of acoating that projects invisible light.
 40. The method of claim 33,wherein the object is selected from the group consisting of a remotecontrol device and a set of keys.
 41. The method of claim 33, whereincapturing the video signal comprises: generating the visible componentand the invisible component with a wide frequency charge-coupled device(CCD).
 42. The method of claim 33, wherein capturing the video signalcomprises: generating the visible component with a first charge-coupleddevice (CCD); and generating the invisible component with a second CCD.43. The method of claim 33, wherein positioning the halo comprisesproviding a halo with a shape selected from the group consisting of acircle, an ellipse, and a polygon.
 44. The method of claim 43, whereinthe object comprises a person engaged in video communication using thecamera, the method further comprising: disposing an illuminator on ahead of the person to project the target of invisible light.
 45. Themethod of claim 44, wherein positioning the halo further compriseshorizontally positioning the halo such that a center of the halo ishorizontally aligned with the illuminator.
 46. The method of claim 33,wherein the object comprises a person engaged in video communicationusing the camera, wherein positioning the halo comprises providing ahalo with a shape that resembles at least a portion of an outline of theperson.
 47. The method of claim 46, further comprising: disposing anilluminator on a torso of the person to project the target of invisiblelight.
 48. The method of claim 46, wherein capturing the first videosignal comprises receiving the invisible light in the form of infraredradiation generated by the person.
 49. The method of claim 33, whereinpositioning the halo comprises providing a halo with a user-selectedshape.
 50. The method of claim 33, further comprising: continuouslyrepositioning the halo in response to motion of the target within theinvisible component.
 51. The method of claim 33, further comprising:automatically establishing a size of the halo based on a distance of theobject from the camera.
 52. The method of claim 33, wherein obscuringthe portion of the visible component comprises gradually increasing alevel of alteration of the visible component across an edge of the halo.53. The method of claim 33, wherein obscuring the portion of the visiblecomponent comprises replacing the portion with a single color.
 54. Themethod of claim 33, wherein obscuring the portion of the visiblecomponent comprises blurring the portion to make objects within thevisible component less distinguishable.
 55. The method of claim 33,wherein obscuring the portion of the visible component comprisesreplacing the portion with an image unrelated to the visible component.56. The method of claim 33, further comprising: displaying at least asubset of the visible component of the video signal on a local displaydevice viewable from within a field-of-view of the camera.
 57. Themethod of claim 33, further comprising: transmitting at least a subsetof the visible component of the video signal to a remote terminal fordisplay.
 58. The method of claim 57, wherein transmitting at least asubset of the visible component of the video signal to a remote terminalcomprises utilizing a network selected from the group consisting of acable television network and a direct broadcast satellite network. 59.The method of claim 58, further comprising: receiving televisionprogramming from the network for display at a location viewable fromwithin a first field-of-view of the camera, wherein the televisionprogramming is received in a set top box that transmits the video signalto the network.
 60. The method of claim 57, further comprising:receiving a second video signal from the remote terminal for display ona local display device.
 61. The method of claim 60, wherein the secondvideo signal and at least a subset of the visible component of the videosignal are displayed simultaneously.
 62. A method for automaticallyobscuring a portion of a video signal, the method comprising: capturinga video signal depicting an object with a camera sensitive to invisiblelight, the video signal having visible and invisible components;processing the invisible component to determine a position of a specifictarget of invisible light projected from the object, wherein the targetincludes at least one characteristic that distinguishes it from othertargets; automatically positioning a halo in the visible component withrespect to the position of the target; obscuring a portion of thevisible component that lies outside the halo; providing a database thatcontains identities and settings for a plurality of objects; receivingan identity of the object from the database; and retrieving theuser-selected shape that corresponds to the identity of the object. 63.A method for automatically obscuring a portion of a video signaldepicting an object, the video signal having visible and invisiblecomponents, the method comprising: processing the invisible component todetermine a position of a specific target of invisible light projectedfrom the object, wherein the target includes at least one characteristicthat distinguishes it from other targets; automatically positioning ahalo in the visible component with respect to the position of thetarget; and obscuring a portion of the visible component that liesoutside the halo.
 64. A method for automatically obscuring a portion ofa video signal depicting an object, the video signal having visible andinvisible components, the method comprising: receiving a position of aspecific target of invisible light projected from the object, whereinthe target includes at least one characteristic that distinguishes itfrom other targets; automatically positioning a halo in the visiblecomponent with respect to the position of the target; and obscuring aportion of the visible component that lies outside the halo.
 65. Asystem for automatically obscuring a portion of a video signal, thesystem comprising: means for capturing invisible light that captures avideo signal depicting an object, the video signal having visible andinvisible components; means for processing the invisible component todetermine a position of a specific target of invisible light projectedfrom the object, wherein the target includes at least one characteristicthat distinguishes it from other targets; means for automaticallypositioning a halo in the visible component with respect to the positionof the target; and means for modifying the visible component to obscurea portion of the visible component that lies outside the halo.